Sure thing. You were discussing some kind of other simple program that you were looking for a name for?

Right, yeah, on #omnimaga. I made a topic there for it on DJ's advice. I might as well make one here too.

Good stuff, I'll go take a look. On a side note, how far would you say that this project has progressed to date, percentage-wise?

alrighty, I have a 89 titanium now, so I can try this baby out

(have to read the readme first )

Some notes on the readme:

>> Could you please credit me as Christopher "Kerm Martian" Mitchell rather than 'KermMartian'? Thanks.
>> It's "Cemetech", not "Cemtech"

Sure, I'll change the way you're credited. And oops. Meant Cemetech, of course.

Do you want me to reupload a new version now, or can it wait for the next release?

Nah, no rush, it can easily wait until the next release. What do you want to do between now and the next release?

Optimization, and maybe more tokens if I feel like it.

The main thing I want to do is "compile" token files so instead of my complicated parsing code I could just do something like this:


Code:


I hope that would make things faster. It would mean that I would need to include a program to compile the token files though (that is, parse the text file and then write the tokenInfo array to another file).

I tried reading that latter half of your post through a few times, but I couldn't really get it to parse. Can you run that by me again please?

At the moment, the token info parser reads a text file into a string (character array) and then sorts through it.

What I would do is maybe write a seperate program to do this, and then write the objects to a new file. I'm not really sure how to explain that- but instead of writing a string (character array), it would write a tokenInfo array (creating a custom appvar format).

Then, when it comes time to set up token infos, I could read the file directly into a tokenInfo array rather than need to parse it.

Does that make sense?

So, another update. I've gotten the "compiled token files" working. The interface for the seperate program is shown in the screenshot.

I decided it might be a good idea to define the file type in each token file (that's what "File type" is), so a file compiled for, say, the 85 could say "85P" in the VAR-LINK menu instead of 8XP. But I haven't implemented that in code yet.

This causes token initialization times to be near instantaneous- under a second. The downside? The size of a compiled token file is about double the size of the text file. So I left the code to parse text files in Solar89, giving users a choice between the larger and faster files or the slower and smaller files.

The actual tokenization process is still slow- hopefully that can be rectified with additional optimizations.

Regarding your previous post, that makes sense, thanks for clarifying. Regarding this latest post, that also makes sense; congrats! I'm glad to hear you found a time-memory tradeoff for either fast startup or small footprint that's selectable.

I've gotten the file type/extension (whatever you want to call it) implemented. This is only a feature if you're using a compiled token file- otherwise, all files will be given the 8XP file type by default.

I've also added code to automatically archive the resultant files- both for my token compiler (on calc-name is maketok) and Solar89 itself.

I added whitespace support, so Solar89 will automatically discard all extra spaces in front of lines. It won't replace them with : characters or anything like that (that's a good way to waste memory)- it just trims them out at the start.

I'm also trying to get empty lines supported, but I'm having some weird issues with that. If the line is at the top of the program, it works fine. If it's in the middle, between two other lines of code, tokenization takes much longer than usual and then it hangs. And if it's at the end, the program runs as normal but the lines at the end don't appear for some reason.

The last problem isn't a huge one (because Solar89 already inserts an extra newline character at the end of the each program), but the first problem definitely is.

Aww, I'm sorry to hear that you're having some difficulties, although it's great that you have that type/extension system implemented. What, other than bug extermination, is left as far as development goes?

A detokenizer, and of course my X-Link converter idea which will be a seperate project. I want to work on that after I finish up the next release, so the detokenizer will be postponted until then.

Then, of course, there's more optimizations (hopefully), and support for more tokens which I really don't feel like doing.

I mutated the bug, though- now, it works but produces two empty lines when it should be producing one. At least I know what's wrong now, so it should be easy to correct.

I've fixed the bug. And I've released 0.3, with versions uploaded to ticalc.org, Cemetech, and Omnimaga. Can't download until they're approved, though.

The full list of changes are:

-Support for binary token files (with included program- maketok- to make them)
-Support for whitespace in text files
-Bugfix where empty lines in the middle of a program caused infinite loop
-Solar89 and maketok auto-archive output files

Renamr and Solar89 have both been approved on Cemetech. Congrats for the new changes; I think you're making an awesome program in Solar89, and I hope a lot people start downloading and using it soon.

Just a few brief updates.

First thing, I made a minor interface change- the tokenizer now outputs each line of the file to the status bar as it runs. I made this mainly to convince the user that the calc hasn't frozen- it's actually doing something. (Since tokenizing can take some time).

Second, I recently wrote another example- a quadratic solver. Just as an example as a slightly longer program (and how much time a mathematical program might be expected to take).

Third, I talked with thepenguin77 earlier today about making a better tokenizing algorithm- a tree based system- to increase speed. I plan to code that when I have the time (unfortunately, not now).