Cross-posted from http://ourl.ca/9886.new#new

While the .g3a format is useful for add-ins, it's bulky and filled with unused space. Furthermore, it greatly increases the amount of memory used for small programs. In light of this, several people, myself included, have come up with new file format proposals to use in an upcoming shell.

I apologize for the extremely long code blocks, but I don't see anything with which to hide them. Attachments with the file formats and a program demonstrating my format can be found linked at the Omnimaga post.

z80man's proposal:

Quote:
.org $30002000
data.l $xxxxxxxx ;header for shell
data.l $00000002 ;size of executable
data.b $01 ;true if MMU is to be reset
data.b $03 ;speed setting, 0-3
data.b $00 ;true if interrupts are to be disabled
data.b $01 ;true if fpu interpreter is to be enabled
data.b $01 ;true if all general purpose registers are to be cleared before execution
data.b $01 ;true if VRAM and screen are to be cleared
data.b $00 ;true if executable space is to be cleared
data.b $00 ;true if to begin execution in priveleged mode
;next 176 bytes are to be used for future expansion
data.l [176 bytes]
;next 64 bytes are for program description. Null terminated
data.b [64 bytes]
;256 color 64 * 64 icon
data.b [HECKTON of bytes]
;executable begins at $30004100
RST
NOP


Josjuice's proposal:
http://wiki.prizmwiki.co.cc/wiki/User:JosJuice/File_stuff

My proposal: Probably the most complex, but also probably the most flexible, since addresses aren't explicitly defined and even changes in the format itself can be accommodated. A sample program demonstrating the format with a 32x32 16 bit color icon is attached below. It was designed to be used with compilers and shells, since the user isn't expected to interact with the format much. The header itself takes up marginally more than 2 KB. This format also has the advantage of providing a robust system against minor errors and making the detection of serious errors much easier.


Code:
Alternative: Uses a chunk based format (hereafter referred to as packets) in order to maintain compatibility with a wide range of data types
File header format:

Types of errors:
   Type 1: Low severity parsing error. The user does not need to be informed and the parser can continue with little to no error handling.
   Type 2: Moderate severity parsing error. The user may be informed if the program deems necessary and some error handling should generally be administered.
   Type 3: High severity parsing error. The user should be informed and all parsing should be stopped to handle the error immediately.
   
Packets are identified by a 32 bit header in order to permit efficient register loading during parsing

Packet Bytes 1 & 2: ID bytes.
   If the ID word is not recognized, then parsers must skip the packet.
   The only time parsers are not allowed to skip packets is if the packets are on a special list (listed below). All programs must contain a copy of this list.
   If the ID is found on the secondary list, the parser must return a type 3 error, since further parsing is either useless or impossible.
      File Type
      FileName
      Main()
      
Packet Bytes 2 & 3: Size bytes.
   The size bytes detail the size of the data without the header and are equal to the data size in bytes.
   This means that if the packet needs to be skipped, then the address of the next packet is equal to the address of the first byte of data plus the size word.
   If the size word is 0x00 00, then the size isn't applicable to the packet type.
   If it is 0xFF FF, then the data size is unknown and a type 1 error should be returned.
   All other words are valid sizes.
   Within the file parsing program itself, each entry in the array of understood packets should include within it the earliest format version that the entry is compatible with.
   If a version compatibility packet is present and the packet requires an earlier version than is available for certain packets, they should be skipped.
   However, if the packet is a critical packet, then a type 3 error must be generated.
   
Types of packets:
   Parent program: ID word is 0x00 00
      Contains a 12 byte hex string designating the parent program. This may be a shell, a compiler, or any other program.
   File type:    0x00 01
      Details general information about the type of file, such as image data or executable program.
      Two byte packet.
         0x00 01: Executable SH3 Machine code (Prizm equivalent of .exe)
         0x00 02: Routine library
   FileName: ID word is 0x00 02
      Contains an 8 byte ASCII file name of the form **** ****.*** where * can be any printable ASCII character.
      The characters following the period are the file extension.
      Program name should be padded with 0xFF bytes if not long enough. 12 bytes.
   Version compatibility: ID word is 0x00 03
      Four byte packet that details the earliest format version that should be applied to each packet.
      If this packet is not present, a type 2 error should be generated.
      The format for this packet is SB VV SS SV, where SB is the byte 0x55, VV is the version number in hex, SS secondary version number, and SV is the sub-version number.
      For example, if the format was of version 01.04.10, then the data in the packet would be 55 01 04 10
   Header packet: ID word is 0x00 10
      Details additional file specific information, such as the author's name, file comments, references to other files,...
      General purpose packet. Can include other packets within itself. Size is thus not applicable. Look for the header end packet.
   Language: 0x00 11
      Contains a three byte ASCII string detailing the language.
      E.G. ENG for English, GER for German, etc...
   Checksum: 0x00 12
      Contains a 4 byte checksum of the data contained by the Main() packet. The checksum is a standard 32 bit checksum taken Mod 2^32
   TimeStamp packet: 0x00 13
      Fourteen byte packet consisting of YYYY MM DD HH MM SS, using the 24 hour clock with zero indexed values on the hour, minute, and second.
   HeaderEnd: 0x01 00
      Marks the end of a Header packet.
      Size not applicable.
   Icon Packet: 0x01 01
      First byte is the number of colors used in the icon image. 0x02 is monochrome, 0x08 is 8 color, 0x10 is 16, etc.
      If the first byte is 00 or another unsupported byte, a type 1 error should be thrown and the packet skipped.
      If the program requires an icon and the first byte is invalid, the program may contine trying to parse the icon or it may substitute a default icon.
      The second and third bytes of the packet are the width and height respectively of the icon.
      Fourth byte is unused except for data alignment.
      Note that the packet itself can only support icons up to 180 pixels on each side, so all sizes greater than this are automatically invalid.
      It is recommended that only certain icon sizes be supported by the program.
      If an invalid size is encountered, it may handle the errors in a similar fashion to an invalid color byte in the packet.
   File size: 0x01 02
      This four byte packet details the size of the data in Main() in bytes. It is not necessary for the format (although the program may require it).
   Clock speed packet: 0x01 03
      Contains a 6 byte field. The first byte states the CPU clock speed in hex, the second byte details the Peripheral clock frequency, and the third byte details the Bus clock.
      The fourth and fifth bytes contain the values to be written to the FRQCR. If overwriting the register isn't desired, the value 0x2000 should be present in these bytes.
      The sixth byte is contains the byte to be applied to UCLKCR. If the register is not to be overwritten, the value 0x20 should be used.
      Remember that to overwrite UCLKCR, the data must be written with 0xA5 in the high byte of the write word.
   MMUCR: 0x01 04
      A four byte packet. If present, the data in the packet should be written to the MMU.
      This packet need not be supported if the application is to execute outside the P1 and P2 memory areas.
   File load: 0x01 05
      Contains a series of ASCII filenames, each 8 bytes long, of the format   ^**** ****.***/
      The ending "/" character is used to designate the end of each filename and the "*" character designates any printable ASCII character.
      The character at the beginning of the string will be "^" if the file is to be loaded into memory and "#" if it is simply to be checked for its presence on the device.
      If one or more of the files are not present, then the program should inform the user.
   Main(): 0xFF FF
      The packet containing all user data (such as code) within the file.
      Size is not applicable.   
   
File structure:
   Bytes 0-4: ASCII representation of the file extension of the format ".***" If not all of the bytes are used, fill the remaining spaces with null bytes.
   All bytes afterwards are contained within packets, which are to be arranged in the ascending order of their ID bytes.
[/code][/url]
Some great ideas there. As a shell author, I definitely want a pointer very early in the header to some relocatable stuff, either several pieces (version, description, icon, filetype, vectors) or a pointer to a memory area where I can stuff those items.
I can see how that might be useful. Added [to mine] Smile
How complex of a file system are we envisioning? Something integrated with the Casio OS or something built on top? How are items found in some sort of VAT, anyway?
Rather than a whole new file format, I'd first look into adapting ELF (it is the formerly the Extensible Linking Format), as if would probably be as easy as adding something like a .casio section to a program binary.
The advantage to using ELF is there are already many tools capable of working with them, and you'd only require a little bit of processing from compiler output to get a file which can be used with the device (eg add your section to the file and you're done).
Oooh, I heartily support that proposal! That would be great in terms of adapting standard toolchains to create our programs.
The Tari wrote:
Rather than a whole new file format, I'd first look into adapting ELF (it is the formerly the Extensible Linking Format), as if would probably be as easy as adding something like a .casio section to a program binary.
The advantage to using ELF is there are already many tools capable of working with them, and you'd only require a little bit of processing from compiler output to get a file which can be used with the device (eg add your section to the file and you're done).


According to what I've seen of the format, there's no native support for icons or physical addresses, never mind important stuff like preset registers and clock settings. Sure, we could modify the format to fit the Prizm's needs, but we may as well just make a new file format, since we'd have to modify all of the tools anyway. Furthermore, the ELF format is rather complex for a efficient parsing, as it's way overdesigned for our purposes. Also, GCC isn't the only compiler I'm thinking about Wink
No native support because sections can contain whatever you want. There are some standard sections (say, .text, containing your actual program code), but you're free to insert whatever you want. Really all you'd need on the prizm is probably just .prizm and .text, which makes for a very simple file to handle.

Aside from that, most toolchains support ELF- I know Renesas' packaged with HEW does, as does Clang and the obvious GCC.

Qwerty.55 wrote:
important stuff like preset registers and clock settings.
I don't see why that should even be needed, unless the SH3 has very odd behaviour. You'd want to expect the environment is clean at program startup (as a C runtime would do), and you might have a preferred clock rate to run something at, but I'd think that would be better left up to the user.
I am definitely down with the ELF suggestion; as Tari says, that can all go in the custom sections.
The Tari wrote:
I don't see why that should even be needed, unless the SH3 has very odd behaviour. You'd want to expect the environment is clean at program startup (as a C runtime would do), and you might have a preferred clock rate to run something at, but I'd think that would be better left up to the user.


Would you happen to have a good reference on the SuperH 32 bit ELF file format that I could look at? And by good format, I mean byte level detail. I'm not inclined to go reverse engineering another file format to supplement already adequate file formats simply to support tools that don't really exist in any significant quantity.

Also, the user is precisely the person you don't want controlling the clock speed.
Just google the elf file format, just about everything supports it so I'd assume the docs must exist.
Wikipedia is of course a good place to start:
http://en.wikipedia.org/wiki/Executable_and_Linkable_Format

I did five seconds of Googling for '"sh-elf" format', nothing yet, but I'm sure it's out therre.
Well there are also elf to .g2a scripts for the fx-9860G/CG which would help you as well.
There's also readelf (part of the elfutils), which can give you detailed information on the contents of a given file. I'd be happy to give it a closer look if someone can get me a relevant file to analyze.
Then it would seem we have a sort of Catch-22 between creating and analyzing sh-elf files. Smile
TheStorm wrote:
Just google the elf file format, just about everything supports it so I'd assume the docs must exist.


That's rarely the case with low level software stuff Razz

From what I've seen, file documentation is generally either filled with high level abstractions that don't tell you much about how the file actually looks or is incomplete. Of course, some things are just obscure and were only published on a personal site a decade ago, so they're nearly impossible to find above the noise of the "content farms."
KermMartian wrote:
Then it would seem we have a sort of Catch-22 between creating and analyzing sh-elf files. Smile

How so? I don't have a copy of sh3-elf-gcc, but surely someone does or has files generated by some other compiler.

Among other places, the FreeBSD manpage elf(5) has good information. Wikipedia knows plenty too.
*ahem. You were saying something about not being able to make sh3 elf files, I've had this for weeks.

http://jonimoose.net/calcstuff/prizm/sh3-unknown-elf/
*bump* What's happening with Prizm development? I want to start writing C programs for my Prizm, and I've seen nothing happen on this lately! Does anyone know if any progress has been made anywhere?
KermMartian wrote:
*bump* What's happening with Prizm development? I want to start writing C programs for my Prizm, and I've seen nothing happen on this lately! Does anyone know if any progress has been made anywhere?
Minor stuff is going on on Omnimaga. Simon and cfxg are working on their "mini-SDKs" and currently Simon is finding some new information about the processor. Parts of it look like an SH-4A!
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
Page 1 of 2
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement