Login [Register]
Don't have an account? Register now to chat, post, use our tools, and much more.
Tokenizer DE v1.0 Released
Published by KermMartian on August 6, 2010 at 2:07:07 AM CST | Discuss this article (36)

In answer to all the users that have requested a custom oncalc TI-BASIC editor, Tokenizer DE takes a first step in that direction. Just as SourceCoder (http://sc.cemetech.net) can convert text files into .8xp programs online, Tokenizer can take text documents typed in Document DE 6 or Document DE 7 and convert them into fully-tokenized BASIC programs editable with the TI-OS's TI-BASIC editor. Weighing in at slightly over 1KB, this small but efficient program takes advantage of the power of the DCS GUI system, and requires Doors CS 6.8 beta or higher. It uses a naive reverse substring-match brute force algorithm that takes very little memory (roughly 300 bytes for the core tokenization algorithm) at the expense of going at a moderate rather than blazing speed. Future improvements may integrate this with a Document DE-style editor; feel free to give it a try and post any comments or suggestions in the associated topic. This release also includes the source code to Tokenizer DE for those wishing to learn more about using the Doors CS ASM libraries.

Download
Tokenizer DE v1.0



Permalink
News Archives Index
2003 2003 [View All]

» January (1)
» March (5)
» April (3)
» May (3)
» June (4)
» July (2)
» August (1)
» September (4)
» December (1)
2004 2004 [View All]

» January (3)
» February (4)
» March (4)
» April (3)
» May (5)
» June (9)
» July (5)
» August (2)
» September (7)
» October (5)
» November (2)
» December (3)
2005 2005 [View All]

» January (8)
» February (4)
» March (4)
» April (15)
» May (21)
» June (20)
» July (6)
» August (8)
» September (10)
» October (12)
» November (12)
» December (8)
2006 2006 [View All]

» January (7)
» February (4)
» March (5)
» April (3)
» May (4)
» June (4)
» July (13)
» August (6)
» September (7)
» October (10)
» November (6)
» December (9)
2007 2007 [View All]

» January (5)
» February (6)
» March (6)
» April (5)
» May (2)
» June (8)
» September (5)
» October (1)
» November (1)
2008 2008 [View All]

» January (2)
» March (3)
» April (2)
» May (3)
» June (3)
» July (1)
» September (1)
» November (1)
2009 2009 [View All]

» January (3)
» February (2)
» March (2)
» April (3)
» May (2)
» June (1)
» July (3)
» August (1)
» September (4)
» October (2)
» December (2)
2010 2010 [View All]

» January (6)
» February (2)
» March (1)
» April (4)
» May (6)
» June (7)
» July (6)
» August (8)
» September (6)
» October (3)
» November (4)
» December (3)
2011 2011 [View All]

» January (7)
» February (5)
» March (8)
» April (3)
» May (4)
» June (4)
» July (1)
» August (6)
» September (3)
» October (4)
» November (7)
» December (6)
2012 2012 [View All]

» January (4)
» February (5)
» March (2)
» April (4)
» May (4)
» June (3)
» July (2)
» August (3)
» September (9)
» October (6)
» November (8)
» December (3)
2013 2013 [View All]

» January (8)
» February (8)
» March (14)
» April (6)
» May (4)
» June (7)
» July (6)
» August (9)
» September (5)
» October (6)
» November (7)
» December (7)
2014 2014 [View All]

» January (3)
» February (4)
» March (10)
» April (2)
» May (4)
» June (5)
» July (4)
» August (7)
» September (7)
» October (5)
» November (4)
» December (5)
2015 2015 [View All]

» January (8)
» February (5)
» March (6)
» April (5)
» May (5)
» June (1)
» July (6)
» August (5)
» September (3)
» October (4)
» November (3)
» December (9)
2016 2016 [View All]

» January (9)
» February (10)
» March (7)
» April (4)
» May (4)
» June (4)
» July (4)
» August (8)
» September (3)
» October (2)
» November (2)
» December (5)
2017 2017 [View All]

» January (4)
» March (6)
» April (4)
» May (3)
» June (3)
» July (2)
» August (3)
» September (3)
» October (3)
» December (6)
2018 2018 [View All]

» January (1)
» February (1)
» March (3)
» April (11)
» May (3)
» June (4)
» July (5)
» August (1)
» September (1)
» October (3)
» November (2)
» December (3)
2019 2019 [View All]

» January (2)
» March (3)

Advertisement