Login [Register]
Don't have an account? Register now to chat, post, use our tools, and much more.
Tokenizer DE v1.0 Released
Published by KermMartian on August 6, 2010 at 2:07:07 AM CST | Discuss this article (36)

In answer to all the users that have requested a custom oncalc TI-BASIC editor, Tokenizer DE takes a first step in that direction. Just as SourceCoder (http://sc.cemetech.net) can convert text files into .8xp programs online, Tokenizer can take text documents typed in Document DE 6 or Document DE 7 and convert them into fully-tokenized BASIC programs editable with the TI-OS's TI-BASIC editor. Weighing in at slightly over 1KB, this small but efficient program takes advantage of the power of the DCS GUI system, and requires Doors CS 6.8 beta or higher. It uses a naive reverse substring-match brute force algorithm that takes very little memory (roughly 300 bytes for the core tokenization algorithm) at the expense of going at a moderate rather than blazing speed. Future improvements may integrate this with a Document DE-style editor; feel free to give it a try and post any comments or suggestions in the associated topic. This release also includes the source code to Tokenizer DE for those wishing to learn more about using the Doors CS ASM libraries.

Tokenizer DE v1.0

August 2010 Articles
August 2010 August 2010 [Back to News Archives Index]

» Doors CS 7 SDK Released published August 4, 2010 by KermMartian
» Tokenizer DE v1.0 Released published August 6, 2010 by KermMartian
» Presenting Document DE 7 published August 10, 2010 by KermMartian
» Doors CS 6.9 Beta published August 13, 2010 by KermMartian
» Doors CS 7.0 Release Candidate 1 published August 20, 2010 by KermMartian
» Doors CS 7.0 Released! published August 25, 2010 by KermMartian
» Contest #7: One Week to Go! published August 29, 2010 by KermMartian
» Updates: Stats, Forum, Mobile published August 31, 2010 by Alex