- App Development Options
- 23 May 2016 11:12:26 pm
- Last edited by DrDnar on 24 May 2016 04:20:54 pm; edited 7 times in total
It has been well over a year since the TI-84 Plus CE came out. On the subject of an app SDK, we've heard about as much from TI as apologies from Donald Trump. The current guess in the community is that the decision to continue to require app signatures is a matter of internal politics; the same person(s) that wanted apps to continue to require signatures also don't want to release the app signing key. It would seem that currently, there is an internal stalemate of sorts, in which the people in TI that are in favor of 3rd-party software convince the people who are not to continue to tolerate our assembly-language antics.
To be clear, there are no sound technical reasons to require app signatures in the first place. The only useful function they fulfill is as an integrity check, which could be done easier and faster with a simple CRC. Otherwise, programs can do anything apps can do, including:
In the past year, the community has made good progress producing our own SDK. We are capable of producing apps ourselves, notwithstanding our lack of a signing key. To the end of supporting app development, I have been working on our own linker that will support both RAM programs---specifically, LibLoad relocatable libraries, in C---, and flash apps. However, on the point of flash apps, the community is not unified. How should we implement them?
There are two ways to do flash apps without a signing key: Fake them using programs in the archive, and patch out the OS's app signature check.
Fake apps would be implemented as special appvars. A special shell would predict where in the archive the appvar would appear when archived, process the app's relocations list while the appvar is in RAM, and archive it. A menu hook can hook into the apps list to make the fake apps appear in the Apps menu.
Pros of fake apps:
Cons of fake apps:
Alternatively, we could patch the OS, a process popularly called jailbreaking on other platforms. Personally, I suspect TI would not take steps to prevent our patcher from working as long as the patcher cannot adversely affect the functioning of TestGuard. To that end, the patcher would need to be highly obfuscated so that inexperienced people could not adapt it to perform other patches which TI would find objectionable. And I must say, in my experience, people with the skills to develop OS patches do not need to cheat on STEM tests. However, patched OSes will fail TestGuard validation. (Certainly, TestGuard validation can be hacked, but developing such functionality also provides great potential for such to be adapted to cheating.) Therefore, we should also provide an unpatcher.
Pros of an OS patch:
Cons of an OS patch:
Before continuing with work on either option, I think the community should have a discussion on which approach to take. I am in favor of the OS patch option, but feel free to discuss your own opinion in detail. Furthermore, this is an open forum thread. I know that some of TI's employees occasionally look at the Cemetech forums. If anybody would like to bring this to their attention so they can see our thoughts and intentions, please do so. And if TI would like to add their opinion, please join Cemetech and talk with the community! We are not happy with this lack of communication. If we are really lucky, this thread will serve to change that.
Regardless of what TI may or may do, the community is not a monolithic entity. Those of us who would like to be on friendlier terms with TI do refrain from releasing any tools that promote cheating, but we cannot guarantee that others will not. Yet programming the OS to enforce app signatures and withholding the key does not change that. There is no shortage of ways to bypass the flash write lock and patch the OS in arbitrary ways. I know that those of us who can write OS patches are not waiting for TI to release the key, and then publish an evil TestGuard breaker. It has been a year. If we wanted to publish such a thing, we would just do it, without waiting. The reason such a thing is not currently available on ticalc.org is that the people who can make tools to defeat TestGuard have no need to do so, because math skills and programming skills go hand-in-hand.
Finally, I am also aware that TI Education is engaged with a cat-and-mouse game with some members of the Nspire series community. However, there is a lot of non-overlap between them and the rest of us. Please do not begrudge us their actions.
Discuss this matter. I hope for a community consensus in about two weeks.
Addendum for TI: Since writing this, I have remembered that Push-to-Test and TestGuard treat apps with certain names differently. I understand the desire to treat certain apps specially. However, I believe the proper way to do that is with a special header field, and to restrict recognition of that field to apps with a special signing key.
On the TI-84 Plus CE, that signing key could simply be the normal one, because that is still secret to TI. TI could release a new OS version that allows the app key field to be absent or have only a dummy value, and for such apps to lack a signature. When Push-to-Test and TestGuard look for apps to treat specially, they can also do a quick check for the signing key ID. This would allow 3rd-party apps without there being a chance of tricking testing tools into thinking they're TI-endorsed in some way.
To be clear, there are no sound technical reasons to require app signatures in the first place. The only useful function they fulfill is as an integrity check, which could be done easier and faster with a simple CRC. Otherwise, programs can do anything apps can do, including:
- install hooks;
- using a hook, display a custom Apps menu, and display our own programs on the Apps menu as though they were proper apps;
- persist after RAM resets;
- be larger than 64 K, by loading additional segments;
- be relocatable, by using our own relocation engine; and
- execute from the archive by applying relocations based on an algorithm that computes, before calling _ArchUnarch, where the program will end up in the archive.
In the past year, the community has made good progress producing our own SDK. We are capable of producing apps ourselves, notwithstanding our lack of a signing key. To the end of supporting app development, I have been working on our own linker that will support both RAM programs---specifically, LibLoad relocatable libraries, in C---, and flash apps. However, on the point of flash apps, the community is not unified. How should we implement them?
There are two ways to do flash apps without a signing key: Fake them using programs in the archive, and patch out the OS's app signature check.
Fake apps would be implemented as special appvars. A special shell would predict where in the archive the appvar would appear when archived, process the app's relocations list while the appvar is in RAM, and archive it. A menu hook can hook into the apps list to make the fake apps appear in the Apps menu.
Pros of fake apps:
- No need for any exploits
Cons of fake apps:
- A special shell is needed to install the Apps menu hook after every RAM reset
- The relocations list needs to be reprocessed after every garbage collect
- The app needs to be split into < 64 K segments. This can probably be handled by the PC-side linker and calc-side relocator, though it will still require programmers to take special steps with read-only data structures that need to be kept together (e.g. bitmaps).
- The shell for fake apps will need to verify that the file still exists in the correct location before running. This is easy: just write the expected location into the file, and compare.
- Archive hooks are unsafe; there is always a small chance (on the order of 0.5 %) of crashing after a garbage collect. That said, whether or not to use flash hooks is up to developers, and for stability, a RAM hook could be used to verify and chain to a flash hook.
- It will be an awful hack that will be annoying to program for, and annoying for end-users
Alternatively, we could patch the OS, a process popularly called jailbreaking on other platforms. Personally, I suspect TI would not take steps to prevent our patcher from working as long as the patcher cannot adversely affect the functioning of TestGuard. To that end, the patcher would need to be highly obfuscated so that inexperienced people could not adapt it to perform other patches which TI would find objectionable. And I must say, in my experience, people with the skills to develop OS patches do not need to cheat on STEM tests. However, patched OSes will fail TestGuard validation. (Certainly, TestGuard validation can be hacked, but developing such functionality also provides great potential for such to be adapted to cheating.) Therefore, we should also provide an unpatcher.
Pros of an OS patch:
- It is easier for us to develop a highly-obfuscated patcher and linker than a hacky shell and toolchain to support fake apps.
- It is probably easier and more convenient for end users. First, the jailbroken apps do not need to be relocated after every garbage collect. And second, the jailbreaker patch can be uninstalled after the user has installed the desired apps; apps are not revalidated after installation.
- It could showcase what we can do with proper flash apps.
Cons of an OS patch:
- It is an OS patch nonetheless, and some community members feel that any kind of OS patch will jeopardize our chances of ever getting a signing key, nevermind their SDK. (Yes, TI, we know you have an SDK. You used it to create the apps that already exist. They didn't just materialize out of thin air.)
- The end-user has to remember to unpatch their calculator before any time the calculator might be subject to TestGuard validation.
- Robust encrypting obfuscation depends on obscure hardware quirks to generate the decryption key, making it impossible to deobfuscate without either a very good emulator, or intimate knowledge of the hardware. However, that means that a new ASIC revision could break the decryption routine. Unfortunately, robust obfuscation requires that you allow crashing if decryption fails, since that makes it harder for an analyst to determine if the code has been properly deobfuscated.
Before continuing with work on either option, I think the community should have a discussion on which approach to take. I am in favor of the OS patch option, but feel free to discuss your own opinion in detail. Furthermore, this is an open forum thread. I know that some of TI's employees occasionally look at the Cemetech forums. If anybody would like to bring this to their attention so they can see our thoughts and intentions, please do so. And if TI would like to add their opinion, please join Cemetech and talk with the community! We are not happy with this lack of communication. If we are really lucky, this thread will serve to change that.
Regardless of what TI may or may do, the community is not a monolithic entity. Those of us who would like to be on friendlier terms with TI do refrain from releasing any tools that promote cheating, but we cannot guarantee that others will not. Yet programming the OS to enforce app signatures and withholding the key does not change that. There is no shortage of ways to bypass the flash write lock and patch the OS in arbitrary ways. I know that those of us who can write OS patches are not waiting for TI to release the key, and then publish an evil TestGuard breaker. It has been a year. If we wanted to publish such a thing, we would just do it, without waiting. The reason such a thing is not currently available on ticalc.org is that the people who can make tools to defeat TestGuard have no need to do so, because math skills and programming skills go hand-in-hand.
Finally, I am also aware that TI Education is engaged with a cat-and-mouse game with some members of the Nspire series community. However, there is a lot of non-overlap between them and the rest of us. Please do not begrudge us their actions.
Discuss this matter. I hope for a community consensus in about two weeks.
Addendum for TI: Since writing this, I have remembered that Push-to-Test and TestGuard treat apps with certain names differently. I understand the desire to treat certain apps specially. However, I believe the proper way to do that is with a special header field, and to restrict recognition of that field to apps with a special signing key.
On the TI-84 Plus CE, that signing key could simply be the normal one, because that is still secret to TI. TI could release a new OS version that allows the app key field to be absent or have only a dummy value, and for such apps to lack a signature. When Push-to-Test and TestGuard look for apps to treat specially, they can also do a quick check for the signing key ID. This would allow 3rd-party apps without there being a chance of tricking testing tools into thinking they're TI-endorsed in some way.