Like making the calculator bootloader check the flashed firmware and display a splash-screen that tells if the calculator is running an official firmware or not. All you need to trust then is that the bootloader hasn't been tampered with and that the RESET button works.
The calculator already does this. The issue is, the calculator entirely refuses to boot if the OS isn't signed by TI, so that means that there are no developer options besides finding bugs in either the boot code or the OS. You also can't trust that the bootloader hasn't been tampered with (as you could always reflash the ROM chip, even though that's pretty difficult), not can you trust that the reset button works (you could wire it up to a particular key, then run a program that waits for that key and fakes a reset).
I've been considering how to make a calculator entirely secure. The best solution that I've come up with so far is to have a USB dongle (or a mobile app that uses NFC) that causes the calculator to enter test mode. To do this, it would send a unique ID for the dongle, test mode settings, and a randomly generated challenge to the calculator. The calculator would then sign the exact message that the dongle sent, plus some extra data about the OS signature and version, with a private key stored in the bootcode and return to the dongle. If the dongle doesn't receive a properly signed response from the calculator within a few seconds, or if the OS signature is incorrect or the OS version outdated, it plays an audible tone to indicate that the test mode entry failed.
If the OS version is incorrect, the dongle could also send an OS update to the calculator. After that completes, it would attempt to verify again.
To exit test mode, the same dongle has to send its ID back to the calculator. To prevent a situation where the dongle could be lost and the calculator stuck in test mode forever, test mode could also be exited by waiting for 24 hours.
This, of course, would be useless if the bootcode was readable, so the flash chip would have to be integrated into the same IC as the CPU, and settings would have to be configured so that only the bootcode can read the key. I'm by no means an electrical engineer, but if I understand correctly this would be fairly easy to do with an ASIC and is probably possible to do with something like an ESP32.
To lessen the impact if a key did manage to get out, each calculator could have its own unique private key, and the private key + serial number could be signed by the manufacturer's private key, which isn't distributed anywhere. That way, if a single calculator's bootcode was somehow read, that key could be marked as invalid through a software update to the test mode dongle/app.
This system would have a number of advantages over the current one:
It's basically impossible to fake a reset. If control isn't given to the bootcode when a test mode request arrives, the dongle will be able to tell and will alert the proctor. If control is given to the bootcode, the calculator will reset properly assuming no bugs in the bootcode.
Running native code outside of test mode or a custom OS version wouldn't compromise security. Since the bootcode verifies the OS, the test mode dongle would detect the custom OS and install a signed OS version only for test mode.
It's not possible to install a modified bootcode version and still enter test mode. Even if you entirely replaced the IC with your own, you would need to know the private key from an actual calculator to be able to enter test mode.
A teacher or exam institution could have fine-grained control over what functions are allowed. Since you don't have to enter test mode details using only physical buttons, the test mode could be set up to disable certain tokens or features without affecting the rest. For example, a high school teacher trying to teach algebra could disable graphing for a test to make sure that students were solving the problems algebraically rather than visually.
This would also allow the calculator to include functionality that current calculators don't due to cheating concerns. For example, it could include an exact math engine or even a CAS that could be disabled in the exam settings. School edition calculators could also include permanent test mode settings to disable this functionality outside of tests for things like homework assignments.
Finally, since program development isn't restricted outside of test mode, developers won't have a reason to try to find bugs that could potentially be used for cheating in order to run their own programs.
This still wouldn't prevent students from connecting the screen and keypad to a raspberry pi and leaving the old bootcode running, but if you're extremely paranoid about that you could probably just make the case out of clear plastic. Not only would it look super cool, it would also allow teachers to visually verify that there are no unauthorized hardware modifications inside.