Login [Register]
Don't have an account? Register now to chat, post, use our tools, and much more.
Tokenizer DE v1.0 Released
Published by KermMartian on August 6, 2010 at 2:07:07 AM CST | Discuss this article (36)

In answer to all the users that have requested a custom oncalc TI-BASIC editor, Tokenizer DE takes a first step in that direction. Just as SourceCoder (http://sc.cemetech.net) can convert text files into .8xp programs online, Tokenizer can take text documents typed in Document DE 6 or Document DE 7 and convert them into fully-tokenized BASIC programs editable with the TI-OS's TI-BASIC editor. Weighing in at slightly over 1KB, this small but efficient program takes advantage of the power of the DCS GUI system, and requires Doors CS 6.8 beta or higher. It uses a naive reverse substring-match brute force algorithm that takes very little memory (roughly 300 bytes for the core tokenization algorithm) at the expense of going at a moderate rather than blazing speed. Future improvements may integrate this with a Document DE-style editor; feel free to give it a try and post any comments or suggestions in the associated topic. This release also includes the source code to Tokenizer DE for those wishing to learn more about using the Doors CS ASM libraries.

Download
Tokenizer DE v1.0



Permalink
2010 Articles
January 2010 January 2010 [Back to News Archives Index]

» Happy New Years from the FreeBuild crew published January 1, 2010 by elfprince13
» [Beta] SysMon Server Monitor published January 15, 2010 by KermMartian
» Bring the Calculators Back to IRC: Reviving #ti published January 21, 2010 by allynfolksjr
» SysMon v1.0 published January 25, 2010 by KermMartian
» Freebuild Linux Support published January 26, 2010 by elfprince13
» One Hundred Thousand Posts published January 31, 2010 by KermMartian

February 2010 February 2010 [Back to News Archives Index]

» TI-84+ OS v2.53 Leaked published February 7, 2010 by KermMartian
» Ultimate Calculator 3 Progress published February 8, 2010 by KermMartian

March 2010 March 2010 [Back to News Archives Index]

» Three New BASIC Programs published March 26, 2010 by KermMartian

April 2010 April 2010 [Back to News Archives Index]

» SAX "Dupe Aborted" Bug Fixed published April 9, 2010 by KermMartian
» Doors CS 7 Development Grinds Sluggishly to a Start published April 16, 2010 by KermMartian
» Promotions; Sonic Screwdriver published April 28, 2010 by KermMartian
» DCS7 Dev Continues; Long-Overdue PwdSE v2.0 published April 29, 2010 by KermMartian

May 2010 May 2010 [Back to News Archives Index]

» DCS7 Dev: TabFuncs, Folder Restore published May 3, 2010 by KermMartian
» Doors CS 6.3 Beta published May 12, 2010 by KermMartian
» BinPac8x v1.0 published May 20, 2010 by KermMartian
» Master's Thesis Completed published May 25, 2010 by KermMartian
» Master's Graduation; DCS 6.4b Coming Soon published May 27, 2010 by KermMartian
» Doors CS 6.4 Beta published May 30, 2010 by KermMartian

June 2010 June 2010 [Back to News Archives Index]

» BinPac8x v1.2 published June 1, 2010 by KermMartian
» Doors CS 6.5 Beta published June 5, 2010 by KermMartian
» SourceCoder 2.5: Updates and Features published June 11, 2010 by KermMartian
» TI-84+ Etch-a-Sketch With Real Shaking published June 12, 2010 by KermMartian
» Doors CS 6.6 Beta published June 19, 2010 by KermMartian
» Cemetech Contest #7: Doors CS published June 22, 2010 by KermMartian
» DCSB Libs Progress published June 27, 2010 by KermMartian

July 2010 July 2010 [Back to News Archives Index]

» Doors CS 6.7 Beta published July 2, 2010 by KermMartian
» Return of the mobileTunes 3 Converter Upgrades published July 6, 2010 by KermMartian
» Doors CS 6.7.6 Beta published July 21, 2010 by KermMartian
» Doors CS 6.8 Beta published July 25, 2010 by KermMartian
» Cemetech Contest #7 Deadline Fast Approaching published July 29, 2010 by KermMartian
» BinPac8x Updated to v1.3 published July 30, 2010 by KermMartian

August 2010 August 2010 [Back to News Archives Index]

» Doors CS 7 SDK Released published August 4, 2010 by KermMartian
» Tokenizer DE v1.0 Released published August 6, 2010 by KermMartian
» Presenting Document DE 7 published August 10, 2010 by KermMartian
» Doors CS 6.9 Beta published August 13, 2010 by KermMartian
» Doors CS 7.0 Release Candidate 1 published August 20, 2010 by KermMartian
» Doors CS 7.0 Released! published August 25, 2010 by KermMartian
» Contest #7: One Week to Go! published August 29, 2010 by KermMartian
» Updates: Stats, Forum, Mobile published August 31, 2010 by Alex

September 2010 September 2010 [Back to News Archives Index]

» Contest #7 Now Closed published September 5, 2010 by KermMartian
» CALCnet2.2 Demonstrates Calculator Networking published September 6, 2010 by KermMartian
» NetPong v1.0 Showcases CALCnet2.2 published September 10, 2010 by KermMartian
» New Global Moderators published September 13, 2010 by KermMartian
» Cemetech Contest #7 Results published September 14, 2010 by KermMartian
» Landmark Months; Contest Downloads; CALCnet 2.2 published September 29, 2010 by KermMartian

October 2010 October 2010 [Back to News Archives Index]

» On the Repair of TI-83+ LCDs published October 3, 2010 by KermMartian
» CALCnet2.2 Manual/Whitepaper; Site Uptime, Downtime published October 8, 2010 by KermMartian
» Upgrades and Downtime; DCS 7.1 Beta Soon published October 29, 2010 by KermMartian

November 2010 November 2010 [Back to News Archives Index]

» Doors CS 7.1 Beta 1 published November 8, 2010 by KermMartian
» mobileTunes v3.2 Released published November 10, 2010 by KermMartian
» Ramping Up to CALCnet2.2 & DCS 7.1 published November 12, 2010 by KermMartian
» Cemetech At 2^17 Posts, Active Despite Crunchtime published November 29, 2010 by KermMartian

December 2010 December 2010 [Back to News Archives Index]

» Doors CS 7.1 Beta 2 published December 5, 2010 by KermMartian
» Doors CS 7.1 Released! published December 14, 2010 by KermMartian
» Merry Christmas, Cemetech and DCS7 Updates published December 26, 2010 by KermMartian

Advertisement