"Tokens" BASIC IDE Project (Pre-2013)

[merthsoft]

I've had an idea for some PC-side calculator related things, so I've been messing around with some ideas. What I have right now is a "detokenizer". That is, it takes in an .8xp file and converts it into readable text. I took the tokens file from Kerm, and modified it a bit for my use. It's not 100% there yet (formatting-wise), but you should be able to load just about any file and read it. I've been designing this to be as extendable as possible, so it'll fit nicely into anything else I make. The Tokens.dll drives most of it. I'll be working on various parts of this on and off. Here's a screen shot:

And here is the program:
http://merthsoft.calcg.org/Detokenize.zip
This form was whipped together in a few minutes, so I realize it's not the greatest, it was just an easy way to show off the detokenizer.


Edit:

You can find the TokenIDE program here as an alternate download: http://tifreakware.net/forum/viewtopic.php?f=47&t=123
_________________
Shaun

[KermMartian]

Looks like you're on the way to something great there, Merth. Good luck with it, and I'm glad the tokens file I gave you is coming in handy.
_________________

[Svakk]

Merth, let me know how this actually turns out. I'm planning sometime in the near future to write a program that will allow the user to write programs using AXE or BASIC in a Visual Studios-Type program. This way the user can execute the program, visualize it using a built in emulator, compile it to .8xp, and open .8xp files to be editited. I would be very interested to see how this works out, since it might make my job a little easier...
_________________

[merthsoft]

This is actually a small step in that direction. This is part of what will be, hopefully, a pretty full-featured TI-BASIC IDE.
_________________
Shaun

[KermMartian]

[quigibo]

How do you plan to convert some symbols that don't really have real unicode equivalents? For instance, the xth root sign and the radian symbol, or do those actually have equivalents?

[KermMartian]

[qazz42]

Yep, Kerm is right, I cant find any too x.x
_________________

[KermMartian]

[merthsoft]

[TheStorm]

[elfprince13]

It would be nice to see this done oncalc to complement the tokenizer.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative

[KermMartian]

[merthsoft]

[elfprince13]

[merthsoft]

Round two: Tokenzier.
I have added the ability to tokenize text. For now the test program is pretty simple, you open a program, and it gives you four text boxes, which are (in order): raw hex, detokenized text, tokenized hex, detokenized text of the tokenized hex. This way you can see that the hex and the text match up.

So here you can see that they match up. Here's a download link if you want to test it out:
http://merthsoft.calcg.org/Tokens.zip
Tomorrow I will be working on taking text that you actually edit yourself and turning it into a program that you can actually put on your calculator.

There are sometimes some inconsistencies, but they tend to be due to ambiguities in the Tokens.xml file, which you're able to edit to suit your needs.

Please let me know, if you test it, whether you find any errors or not. This will, hopefully in the future, be integrating with SourceCoder so that you can post from the program. I'm in talks with Kerm about that.

Also, Kerm, could you change the name of this topic to "Tokens" for me?
_________________
Shaun

[qazz42]

Wow, this looks cool, we now have a TI-BASIC editor for out computer kinda
_________________

[SirCmpwn]

We already have a couple of TI Basic editors out there, like TI Coder, but it will be quite nice to see something new for once.

[merthsoft]

[SirCmpwn]

I actually have a pretty extensive .NET library for mucking about with calculator files. It can create, read, and sign .8xp, .8xu, .8xk and .8xi. It can also convert an 8xi to a bitmap, which is nice
I should publish it at some point, it's d@mn useful.