TI Data-Base

[iteration69]

You should know by the title what this is.

A while ago i worked on project that acted as another calculator and allowed one to control and gather data with a calculator using basic, assembly or what have you.

Well, this is pretty much an extension of that project. At first i started re-writing the project on an Atmel AVR microcontroller. Now that i have the fundamentals done(physical link, byte, packet) I plan to implement an SD card driver.

I should be able to hide the micr and sd card inside the calculator so that it is not cumbersome. I bought a ti keyboard on ebay so that i could have a decent platform to prototype everything on. Basically hide all the electronics in the back of the keyboard adapter housing. For those of you who do not know of the Ti keyboard check it out. I am speaking of the plastic shell that the TI slips into with back stand. I'm using this becuase it allows me to transform my calculator into a very powerful test tool. Keep in mind that my calculator has a front diffused screen light, the technical details on that aspect can be found elsewhere. The calculator in combination of screen light, plastic shell with back stand creates quite the tool indeed.

While working on the project I amused myself with the remote control features. I found these tinkerings to be very practical in that i can sense when the calculator turns on and effectively take control of the system without user interaction. I simply emulate keystrokes and request a program. At one point I wrote a quick assembly program to demonstrate how easily i could hijack the calculator through the link port. This opens the doors to alot of custom applications on models that do no have flash memory.

Back to the Data Base side of the project. We all know that even the most modest SD card is quite large in comparison with the calculators storage facilities and very cheap. I always thought "Gee, wouldnt it be great to have a flash archive on the Ti85,86,82.." Well, that may just be the case in a few months.

Here is my disclaimer.
This project is not for sale.
It will be do it yourself or find someone to do it for you.
I have no problems releasing my source code, schematics, and PCB layouts.



At this point I am considering a group project. So how many people would be interested in something like this?

[magicdanw]

This sounds very cool! I don't know much (ok, anything) about hardware development, but I'd be happy to help with the z80 assembly coding. Let me know if you're interested in the help.

[iteration69]

magicdanw,

If you can handle interfacing to TIOS and the VAT properly i can use the help.

The platforms of focus are TI86,85,82. TI86 being the highest priority. But i do have the other calculators.

Obvioiusly a shell will need to be use on the 85 and 82. I would rather have a community "hash" out so to speak on this aspect.

I currently have plans for both assembly and basic interfacing.

[magicdanw]

I believe I am rather knowledgeable on the TI-83+ OS and VAT, and the other calculators are probably similar in many respects (or I can research the differences). I did quite a bit of work on CalcUtil, an application that further blurs the difference between RAM and archive, which may prove relevant here.

[Taricorp]

Sounds like a worthy project (rather similar to an idea I had, of attaching a flash drive internally to a calc (switchable to enable it, of course)).
I might tap into the link port lines, add a switch to enable it (and disable the link port), and put all the electronics in the space behind the LCD (there's a good amount of space in there, unless your other mods are in there, too- and you'd need to use a QFP-or-some-such package for your AVR (not PDIP, anyway)).

[iteration69]

Taricorp,

My approach will be completely software. The internal device will rely on known TIOS time-outs. This will enable the calc to still use the linkport when needed. When no external device is attached the internal controller will detect the near time out condition and take control of the packet interface.

Other than that, pretty much the same over-all goals.

magicdanw,

Sounds good.


Tari. Would you like to contribute to this project?

Anyone else? Lets make this happen.

[Taricorp]

I'm happy to help, to some extent. I'm rather handy with assembly, but I'm not as familiar with the internals of TI-OS as others here.

Quick idea- create bogus VAT entries for variables which are stored on the card (say, set the top 4 bits of the page entry- use the bottom nibble to extend the address entry to 20 bits maybe). Then install a parser hook which intercepts those and either (a) read the variable from the card and parse as usual, or ( prompt the user to plug the card in, then exit gracefully if they choose not to.

[magicdanw]

[iteration69]

I plan to use FAT on the SD card. The SD card with interface with the microcontroller via SPI. This is the slow mode but i don't think that will be a problem.

So the micro will have drivers for the SD and the calc link port.

My general idea was that a custom assembly application would be developed to interface things. Nothing fancy but in the same token it should have a intuitive user interface. I was basically thinking that this interface program would "browse" the SD data base. The user simply selects what they want from the archive and it handles the request apropriately.

I see somewhat of a problem when it comes to basic interfacing. My goal is to have a device that can interoperate between TI Basic and Assembly. Research needs done on the basic end of things.

At the moment i interface with the device using Get(), Send(). On the ti 85 that is Input "CBLGET",Var and output "CBLOUT",Var, IIRC.

I currently only have support for Real and String type variables.

What are your thoughts so far?

[magicdanw]

Once the program/application is written, interfacing with basic shouldn't be too hard. If the driver is small enough to fit in a program, then a basic program can either run the assembly program normally and let the user transfer variables, or if it wants to transfer a specific variable, it can call the assembly program with some parameters.

If the driver is so large that it needs its own application, then hooks can be used to let the basic program call it. Everything else is the same.

Now, you say that you currently interface with Get() and Send() commands. Does this mean that you've already built the device, and it works with sending variables? Because that would be great! Also, does this mean you're designing the device to use TI's protocol and work with the Get() and Send() commands? If so, that's pretty cool, but I'm not sure it's optimal. Applications certainly can't be send and received this way, and i'm not sure about programs, but I doubt it. And those will be the two most important things people will want to back up with this device, I'm sure.

[iteration69]

The device exists right now. I originally designed it around a zilog ez8 flash micro.
I decided to rewrite everything using an Atmel AVR since there are alot of people who are familiar with the core.

Anyway, the device uses Get() and Send() like mentioned. It is 100% compatible with TI's link protocol. The calculator believes that it is linking with another calculator. This allows me to write programs in basic that can control external hardware. It is no where near efficient. a slow, to be honest.

For example if i want to get data from say channel 1 of the 10bit ADC. I send a few commands, fetch the data, process the data (which is in itself very slow) and then display the data as a waveform. (Temperature log in this case)

The over all through put is about 1 sample every second. This comes from the complex linear interpolation overhead to accurately convert thermistor data.

Without the processing i can hold about 20 samples per second. Still dog slow, but most of the time i don't worry about speed. The abstraction of basic solves alot of problems.

I've discovered alot about how TIOS handles variable request. For example, i could request a real type variable in the following way
Get(ABCD)

I can actually send a variable back named HACK, that is a string (or program) type.
And then i can enter remote control mode and execture that program.

Now, obviously the device needs a special high speed interface mode for the assembly application. I'll need to work on that end. Nothing to it really other than building the protocol.

I like the idea of using the assembly program as a tier. But i also want to keep direct compatibility with TI Basic for other reasons. I see no problems implementing all methods.

[magicdanw]

Well, I must say I am quite impressed! That sounds like a quality piece of equipment you've got there. Bravo! When you first posted this, i thought it was just theoretical, but you've really got most of it done. The assembly should be a piece of cake, but the basic interface you've got right now sounds really good.

I'm not sure I understand how you have a real variable named ABCD or a string named HACK, since both types have 1-token names, but I trust you know what you're doing, so I'm probably just misunderstanding what you're saying (it is 2 AM, when these things tend to occur).

Edit: Oh, I see. You have an 85/86. Those have the cool variable names, i understand. Do you know if your device would work with an 83+ series calculator? (That is what most people on this forum have, including me.)

[iteration69]

You are correct in that the 85 and 86 support more flexible naming conventions. This is why i am able to use elegant names like abcd, and HACK.

I've somewhat tested the ti 82 (ti 83 = beefed up ti82)
the ti 82 has a alot more limits put to place on the variables as you know, and the packet structure is a little different. But i believe i could make it work if need be.

Added:

Looking over the differences between the 85/86 and the 82/83/83+

The 82/83/83+ uses
1 bytes Base-10 exponent
7 bytes Mantissa

and the 85/86 uses

2 bytes Base-10 exponent
7 bytes Mantissa

Wow, what a precision loss on those lower models.

I can fix the code to work with this. But things brings about a general question. Why do you guys use the 82/83/83+ ... What makes it so special?

[magicdanw]

[iteration69]

Your precision loss nightmare is my reality. I've been forced to write basic applications to extend the precision for some statistical forecasting (logarithmic in nature).

At any rate. I need to know if there are any fundamental operating conditions between the 82,83/83+. I have no experience using the 83 series but i have plenty with the 82. My understanding is that the 83/83+ are upgraded (in terms of functions, applications, and speed) versions of the 82. If this is correct i should be able to prototype with the 82 and expect to simply drop an 83/83+ in place of the 82 without adverse operating afects.

The packet interface between 82. 83/83+ looks to be the same though I admit that i have not studied things at this point.

Worst case I'll pick up an 83/83+ for testing purposes.

You say that you have a semi-working 83. What does that mean?

Speaking of programming on the calc side of things. If you have not already done so it would be a good idea to brush up on the link io. I'm sure TIOS has routines for getting and sending bytes. You would know better where to look for those and how they may affect the OS in a custom application. Since I work with basic i don't need to worry much about these functions on the calc. I did write a small program in assembly that directly controls the link. It completely bypasses the OS by disabling interupts and bit banging. There are portions that use some of the lcd functions for putbyte. I simply restore the link state to default and restore the interupts so that the OS does not detect link IO activity.

Though i have a packet interface i'm not sure the assembly program should rely on this interface due to the inefficiencies in TI's packet implementation.

Keep in mind that work still needs done on the controlled end up the project. I need to write a driver for the SD card and the FAT file system. I've done quite a bit of data recovery work on FAT12/16/32 systems in the past so this aspect is not much of a challenge. I still need to develop an SPI driver for the SD card.

Back to the IO. I think the fastest way to develop and test this application would be to emulate it on another calculator or use the linkcable and write an application on the pc(untill the controller is done).

Any thoughts?

[magicdanw]

I think the 82 and 83/83+ are similar in link protocol, because I'm pretty sure you can transfer variables between them. You should take a look at this guide, if you haven't already, which documents ti link protocol and variable formats for pretty much every calculator:

http://www.ticalc.org/archives/files/fileinfo/247/24750.html

I have a semi working 83+ (not 83, they're different mainly in that the 83+ has flash archive space, and the 83 doesn't, but there are some other differences). First it's link port broke off. I managed to solder it back, but sometimes it's a bit finicky, and you have to jiggle or hold the plug a certain way. Next, at least one of the buttons (the plus button, I think) stopped working. Finally, while testing some code that messes with flash, I corrupted the OS, and I'm not sure how easy it'll be to reload the OS, considering the link port's state. But I can try if you'd like to test anything with it. But this shouldn't be necessary, because I also have an 84+SE, which is essentially an 83+ with more flash, ram, faster, etc. The CPU can be told to go at the 83+ speed, so it'll act exactly the same.

The protocol you're using now is pretty good, since you can request variables and send them in basic, meaning that no assembly interface is necessary (although it would be more convenient, of course, when you want to browse what's on the SD card). The OS has routines to send and receive variables using the TI protocol, so that could be used, but of course we could come up with a faster protocol to use as well.

[c_plus_plus]

[iteration69]

Looks like I'll be working with the old 82 this weekend.

The goal will be Real and String types by sunday night. It looks like the 82 lacks remote control commands so there will be a little user intervention in order to use the archive on that model.


I guess at this point you could start building a user interface. And tinkering with abstracted GetByte(),SendByte() routines.


I'll post my progress in a few days.

[iteration69]

I started testing the 82 and 83.

The packet protocol is about 99% compatible with my existing implementation. The 82,83 generally use 1 byte less. I've managed to make the packet interface auto-detect. It uses the machine id byte and adjusts things appropriately.

So, i test an 82,83,85,86,89,92.

I failed the test on the 89,92. Failed so badly that my code actually crashed and dumped 0xC0. Obviously i need to do more research on those models.

I have no problems linking with the other though. I did find a way to break things by going into the calcs manual transfer menu and attempting to send random variables. Apparently the protocol changes a bit when using this manual mode.
Some work still needs done in this aspect.

I still see the 82/83/84 as a very limited machine. A general break down is that we only have single byte variables. Though it looks like real and program variables can share the same name. I need some one to verify this or i can do it myself at some point. I personaly don't think the 82-84 models have much potential but if you want to use one of those I'll attempt to find a way to interface basic. I gotta rant a little about those models and wtf would anyone suggest their use. Maybe i'm just use to my robust ti86. Any way, i tried to use the 83 last night and found it pretty much worthless. While griping to myself about it i noticed the screen doesnt even have a view port protector. This calc was not designed for the long haul like some other models.


<End Rant>

At this point i still need to get an 84 for testing purposes.

My current code base has support for GetByte,SendByte, GetPacket, SendPacket, SniffByte, SniffPacket. The later functions are wrote for stealthy link sniffing to see how the calculators actually communicate when using Get(), Send().

I probably stated before that the protocol for the basic interface is not quite the same as it is for PC to calc. It is close, but no cookie.

I should arbitrary control in a few days. One of the strengths of the device is that the basic program can emulate external functionality even though the existing firmware does not natively support the given function. Basically, i've given TI-Basic the ability to PEEK and POKE bytes to Flash, Sram, and Hardware of the device microcontroller. Though it is very slow it enables rapid prototype development.

Here is a sysnopsis.
One example would be writing code that dynamically configures a timer for pulse width modulation of a given duty cycle. Or acquire some analog set of data points and apply statistics. You can't do that with a few kilobytes of flash and a few hundred bytes of sram. Or maybe something as simple as read an I/O port perform a given function and set an I/O port. Though the latter is very possible it requires the microcontroller to be reporgrammed every time a function or I/O configuration changes. Using a calculator as a host does not require such.

I admit this does not directly relate to the concept of a data base on an SD card but i want this funcionality to be included with the micro incase someone has an advanced need.

I've thought about using a program to interface things. Pretty much an abstract set of commands that can be typed into the TI program editor and then sent. But this creates the problem of, how do we send it. If it is a reserved program name i could simply fetch that name, and then parse it. At this points things are still very much open.

Any thoughts or suggestions on how we interface the lower end models?

[magicdanw]

Let me see if I can summarize some answers that might help you. Yes, the 89/92 is radically different from the other calculators, due in no small part to the advanced CPU and complex, symbolic-oriented operating system, I'm sure.

With the 83 transfers, are you talking about silent link transfers? Those are ones where you can send and get variables without doing anything on the calculator end. That is a different protocol from the protocols used in the link menu. Also, note that the 83 calculators can't send variables to another calculator in Basic. There are two commands to interface with a data device (Get and Send) and one command to interface with a calculator (GetCalc). I hope that helps with something.

Also, the 84+(SE) uses exactly the same protocols as the 83+(SE), right down to the machine ID (the identifying byte in link protocol). So if you get it working on an 83, it's a short step to 83+, and then you're all set.

And don't worry, you're not alone in feeling that the 83+ series is limited. It's just that so many people have it, we have fun trying to stretch the limits. Sure the calculator can't work with programs that are archived - so we wrote shells and other programs that can. And sure the calculator can't do symbolic math - so we wrote an application that can. Plus, it really impresses the math class :)

I'm gonna be away in California for a week, so I might not have internet at the hotel. But keep up the amazing work!