After being directed to this topic a few days ago, I decided I should try my hand at writing some sort of installer to simplify the installation of large BASIC projects.

As a user, you'd send the compressed file(s) to your calculator, followed by a relatively lightweight installer. From there, you run it as an ASM program and it'll give you options of installing, uninstalling, or repairing some large BASIC game. Planned is the ability to install to DCS file folders, either pre-existing or newly-created.

As a developer, you create the compressed archive files using a tool which consists of a batch file (the thing you actually run), some custom-built C program, an pucrunch. All of it will be included whenever I finish up this project and test it for bugs. For creating the archive, you have three folders, each named "RAM", "FLASH", and "MAIN". You put files that go into FlashROM (Archive) in the FLASH folder, subprograms that go in user RAM in the RAM folder, and the file the user ought to run into MAIN. The MAIN folder must contain one program. The other folders in any combination may remain empty, but they must exist. Any file ready to be sent to the TI-83 Plus (i.e. .8xp, .8xs) can be included in the archive. EDIT: Don't try to include group files, though. That kinda defeats the purpose of the installer, and ... well. It might complicate things if you somehow tried to place a group file in RAM.

I've written down the planned specs for these archive files, and the methods I'd be using to put them all together. It won't be pretty, but I'll get it to work. The only thing that would really rack my mind would be trying to understand the pucrunch decompressor well enough to modify for ... special purposes...

Oh, and as far as a GUI goes... there won't be much of one. Gotta go with the "retro" ASCII look. Homescreen ftw Razz
You could always play around with the DCS GUI while you're at it. Smile But I respect your decision to go with a simpler homescreen ASCII GUI. I definitely will be following this project with keen interest, and I hope you'll ask me for any help that you need with DCS-specific stuff, coding, testing, or optimization. Do you think you might consider allowing on-calc creation of these as well for those who don't mind having non-compressed archives and would prefer not to get their computer involved? I look forward to seeing your ideas and progress.
May I suggest using libti for to automate the sending of the files to the calc and even in the file creation/compression. For the sending look at tiput.c in Ben Moody's TiTools as it is a very simple implementation of doing just that.

If you have questions about libti* both me and Lionel would love to help.

Good luck with this it sounds like a cool project.
Mmm, I think that would be a very interesting collaboration indeed. Also, Iambian, please don't hesitate either to post C questions in this thread, or to start a separate thread for C questions.
So I'm having a problem with the following code. It's supposed to grab information out of a file which contains filenames. They're in a directory which I put before the filename, and the string is constructed. It displays on the prompt correctly, but it does not want to open the file. I verified that the filepath/filename combo works with a static variable, but not this.

Sorry if my verbiage isn't right. I am still learning the basics with the whole C thing.

EDIT: All the variables were declared appropriately earlier in the program.

EDIT2: Please disregard this post. I changed fileread[i-1]='\0'; to fileread[i-2]='\0'; and it solved it wonderfully. Something about leftover nonprintable characters, I think.

Code:
        for (numlines=getnumlines(infile);numlines!=0;numlines--) {
            switch (fcounter) {
                case '1': strcpy(fileread,"FLASH/\0");i=6;break;
                case '2': strcpy(fileread,"RAM/\0");i=4;break;
                case '3': strcpy(fileread,"MAIN/\0");i=5;break;
            }
            for(ch=' ';ch!='\n';) {
                fileread[i] = ch = fgetc(infile);
                i++;
            }
            fileread[i-1]='\0';
            for(i=0;fileread[i];putchar(fileread[i]),i++);
            putchar('\n');
            tifile = fopen(fileread,"rb");
            if (tifile==NULL) printf("FILE COULD NOT OPEN\n");
            if (tifile==NULL) break;
            printf("File opened\n");
            if (fclose(tifile)!=0) printf("Sumthing wrong!\n");

        }
Sorry I was so slow to catch up on this post. In the few examples of your coding that I've seen so far, you've written some of the most creative and unorthodox for() loops that I've seen in a very long time; I'm curious what your inspiration is. I believe you need the extra -1 because you increment once again even after getting the \n? If you're testing on Windows, you should be aware that the newline is \r\n, so the "\r" is the other thing you're throwing out.
Thanks for that little tip regarding how newlines are stored in Windows.

I got my inspiration from the following document: http://www.iu.hio.no/~mark/CTutorial/CTutorial.html
It's where the line starts with: for (ch = '*'; ch != '\n'; ch = getchar())
I took that and flew with it Smile
Gotcha, very nice. For the sake of cross platformedness, if you decide that you want to try to achieve such a goal, I suggest being able to recognize \n, \r, and \r\n as valid newlines, for *nix, Mac OS, and Windows respectively.
I may eventually go cross-platform, but right now, I'm just writing stuff that'll work happily with my Vista, which is probably a good reason to have other people test it on other Windows systems. Not entirely sure what differences the versions might have, but they might affect something.

So far, I can get my batch file/C program/pucrunch set to generate one compressed .bin file. Gotta write more code in to have it be able to generate multiple files, and then I've gotta pack it into the .8xp file format, which I will write into mah C program.

Would you guys rather have appvars instead of programs?
Yes Appvars would be much preferred to programs assuming the program to install/unpack them is included. Though from the original post I had the impression that the files were gonna include the code to unpack themselves.
I guess I did not make it too clear. When the install gets to the calculator, there are at least two files. Maybe at least 3 if I choose to write a loader for Athena.

- The huge as hell data files will contain all the information needed for an install. There may be more than one depending on how large the install is.
- Some ASM program (Athena) between 1-2KB will be the installer program.
- Some really lightweight loader program to load Athena to safeRAM so we can have as much RAM as possible available for the install.

I ask for the option about appvars because this is the stage of the project in which I can easily do it.
Wouldn't be a matter of one byte/line in both your C code or ASM program to switch between them, I mean I don't see why you couldn't just support both?
It's too bad that Doors CS' AP system needs to keep the program and its file in RAM, otherwise you could use that. Smile Ah well, keep up the good work.
KermMartian wrote:
It's too bad that Doors CS' AP system needs to keep the program and its file in RAM, otherwise you could use that. Smile Ah well, keep up the good work.


I could write up a small, lightweight program that loads the actual installer into some stretch of saferam. Maybe then I could use some of the facilities that are included with DCS. Otherwise, I plan on writing something that'll function as a standalone installer. If I make both, the choice will be with the game developer. It would benefit them greatly if they were using the DCS libs if they got an installer that would work well with DCS too Smile

I mentioned this on IRC, but I guess I should mention it here too. I got the PC side of the installer nearly done. Well, it's still a small way to go. It'll output a number of compressed files given every 64KB chunk of data input into it, instead of just one file and simply refuse to process the information after 64KB (since the compressor will only process up to 64KB at a time). The end result? I am able to cram Illusiat 13 (130KB) into three files which collectively total 42KB. Still... what needs to be done is to pack these compressed binary files into the .8xp format. I'm not gonna use something like devpac8x since it'll only work on 24KB files. This is a concern because these files (especially if they don't compress well) can reach up to around 64KB. The size itself isn't really a problem since the calc will (believe it or not. I tested this...) accept files up to that large into FlashROM. You'll never be able to unarchive them, but that's the point! Stuff like that should stay in FlashROM.

Aaaanyway. Enough rambling on. If that was too long to read, I just need to finish up the packager, then write the installer (which shouldn't be too hard). Something that works. Then I'll start putting some of the good stuff in. I'll keep ya posted on it.
Have you tried using my BinPac8x? I don't think I coded it with a limit lower than 65535 bytes in files. I'm very happy to hear that this continues to progress; keep up the good work.
KermMartian wrote:
Have you tried using my BinPac8x? I don't think I coded it with a limit lower than 65535 bytes in files. I'm very happy to hear that this continues to progress; keep up the good work.

Thanks for the offer, but I think I'm good at this point.

I have reason to believe that the Athena packer is done (enough). More could be added to it as I get more suggestions, or add more features, but I think it's time to work on the Z80 ASM side of things.

EDIT: Scratch the doneness part. Just experienced a major oversight in the file formatting. I didn't have a way to uniquely identify install files nor did I have a field that one could input a project description. Whoops.
May I strongly recommend that for the case of future compatibility with something DCS based you choose a header format that coincides with the AP file header? For some reason I don't seem to have that in the SDK. O_O No wait, there it is:


Code:
   .org datastart
   .db $BB,$6D      ;required
   .db $C9         ;don't let the TI-OS run this
   .db $31,$80      ;this is an AP file
   .db 0,1,4      ;3-byte type code (see Registered Filetypes)
Start:

You could easily make the first field in the file a description. Of course, you're equally welcome to decide against this. Smile
I'm seriously thinking about making this thing DCS-compatible through the AP system, but as the project stands, that's impossible. As it stands, of course. The files that are associated with the Athena project can be far too large to cram into RAM, and besides that, RAM needs to be rather empty for this to work. So, I'm thinking of a slightly different way of doing it. Not just because I'm planning to make it work for DCS, but because I believe that this will make the project better and easier to work with.

I'm going to split the "main header" off from the first huge data file and put it in its own, rather small file (40-100 bytes). Instead of real names, the data files will get some name that's generated from something like its checksum or hash. Something that won't likely collide with some other file (though in any given project, this will be checked for). These names are stored in the small file so Athena can have a reasonable way to look them up. This will be the file you'll open up in DCS, and if you run a standalone version of Athena, these are the files that are read to determine how many projects there are.

Not exactly sure what you're supposed to call that little file. I'm sure there's a name for it, but I don't know it. Contains information about the install and that sort.
Perhaps "metadata" is the word that you're looking for? It's not 100% accurate, but it's the closest that I can think of. At any rate, I think this would be a great way to handle things, and I obviously love the idea of making it work with DCS/
I believe I have the Athena packer generating files correctly, so I guess it's largely a success?

For anyone interested, here's some documentation about the files themselves.

Code:
-------------------------------------------------------------------------------
Documentation regarding the package file format for the data
portion of the TI-83/84+(SE) installer software. All instances of multibyte
numeric variables are stored little-endian.

Installations using the Athena packager/installer will contain at least two
files in addition to the installer: (1) Metadate file (2) Data files.

Metadata files contain information about the install. This includes, in no
particular order, the number of data files there are, their filenames, the
number of files that are archived inside the package, the estimated amount of
FlashROM the install will use, the amount of RAM it will use, and a slightly
longer (14 character) description field.

Metadata file structure:

Ofs Siz Desc
+00 08: header. $BB,$6D,$C9,$31,$80,$00,$01,$09
+08 01: number of installer files (64KB chunks)
+09 02: number of files to be installed (can be quite a few!)
+11 03: number of bytes of FlashROM the installation will take up
+14 02: number of bytes of RAM the installation will take up
+16 01: Install type flag (not used yet)
+17 14: 14 bytes ASCII text project project name
+31 01: RESERVED
+32 --: Names of installer files (all 8 char long). This is sized
        as number of installer files times eight.

The data files that come after the metadata file contains a number of streams
formatted in the following way:

+00    1b: filetype + flags. Matches TI-OS file types, except bit 7: 0=In RAM
+01    2b: checksum. Used for comparing installed files with this one.
+03    2b: file size (includes possible filesize field that comes with file)
+05    1b: filename length
+06    nb: filename
+06+n    : data stream

The number of these streams will not be known until EOF is encountered. The
amount of data in this file can be up to about 65500 bytes. None of the data
streams are split for the purpose of extracting data from the file. These data
streams are pucrunch-compressed as a solid archive, so it will be up to the
reader to decompress only bytes at a time and then read the information for
its formatting and its EOF.

The name of these data files is derived from the name of the project and the
uncompressed contents of the file. Every file name will be of the form:

Zfffnnnn
Z is just the letter Z
fff are 3 characters from 0-9 A-V. Hashed from name, so will remain constant.
nnnn are 4 characters from 0-9 A-V, hashed from data, so will likey vary.

Examples: These are true examples from an experiment done on packaging
Keven Oullet's (DJ Omnimaga) somewhat incomplete Illusiat 13.

On entering into the command line:
athena -a I13 "Illusiat 13"

I get the following files out:
I13.8xp          Size (bytes) : 132
ZNG0H7DI.8xv     Size (bytes) : 24716
ZNG0LK35.8xv     Size (bytes) : 4846
ZNGOLRSF.8xv     Size (bytes) : 11936

The input was sized collectively at 138327 bytes, so this is also a great
compression ratio. Understand that the actual filesizes on-calc are going to
be a slight bit smaller since calculator files on the computer contain
additional information that is useful for the computer to calculator linking
program.

-------------------------------------------------------------------------------
Documentation regarding the creation of DCS folders

Not created yet.
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
Page 1 of 3
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement