TI-Utils: CLI tools for manipulating calc files on your PC
» >
» Goto page 1, 2, 3  Next
» View previous topic :: View next topic  

What should I name this project?
tisfk (Ti Swiss File Knife)
 22%  [ 2 ]
tiutils
 44%  [ 4 ]
tigrp/tigroup
 0%  [ 0 ]
tifileutils (Ti File Utils)
 33%  [ 3 ]
Other (please specify in thread)
 0%  [ 0 ]
Total Votes : 9

  • TI-Utils: CLI tools for manipulating calc files on your PC
  • 29 Aug 2010 09:27:48 pm
  • Last edited by TheStorm on 11 Feb 2011 09:54:45 pm; edited 3 times in total
A conversation in #ti got me thinking about how there is a lack of tools for working with the different TI file types from the command line. The libti libraries have so many fuctions that could help TI-Basic and ASM programmers alike that it seems like a waste that they are not accessible to userspace for scripting and basic file management.

Ti-Utils is a suit of simple single purpose programs to change this.
The first utility I'm starting on is tigroup/untigroup. I plan for it to be an equivalent to tar but for tigroup files, allowing you to create, modify and extract group files in both the .**g and .tig forms.

Below is the current code, keep in mind this is the first C program I have ever written from scratch, and besides my work on TiLP I have absolutely no C experience. If you have any suggestions or comments please post them here.

Code:
#include <stdio.h>
#include <stdlib.h>
#include <getopt.h>
#include <tilp2/tifiles.h>

int
main(int argc, char *argv[])
{
    int c;
    int digit_optind = 0;
    static struct option long_options[] = {
        {"help", 1, 0, 'h'},
        {"version", 0, 0, 'V'},
        {0, 0, 0, 0}
    };
    char *ofile, *ifile;
    int this_option_optind = optind ? optind : 1;
    int option_index = 0;




    while ((c = getopt_long(argc, argv, ":hVf:o:", long_options, &option_index)) != -1) {
    switch (c) {
        case 'V':
            printf("This is a test.\n");
            break;
        case 'h':
            printf("This will be help text someday.\n");
            break;
        case 'f':
            ifile = optarg;
            break;
        case 'o':
            ofile = optarg;
            break;
        case ':':       /* -f or -o without operand */
            fprintf(stderr,
                    "Option -%c requires an operand\n", optopt);
            //errflg++;
            break;
        case '?':
            fprintf(stderr,
                    "Unrecognised option: -%c\n", optopt);
            //errflg++;
        }


    }
    if (!tifiles_ungroup_file(ifile, NULL)) {
    exit(0);
    }



    return 0;
}

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
That looks great, Jonimus; I'm very much looking forward to seeing how this evolves. Your use of getopt is somewhat ... non-traditional, but if it works, it works. As partially-discussed on #cemetech on IRC, will you be adding ncurses or GTK-based tools, or making pure CUI libs that can be scripted and piped together?
Geopipe
These are intended to be entirely CLI tools that can be scripted and piped together, assuming libti* handles the pipes properly which I'd assume they would.

As the the usage of getopt_long, its almost entirely copypasta from the docs I was using so yeah...

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
Ah, because it's getopt_long stuff, which I was completely missing for some reason. Shock I think I need to have some food or some caffeine (or some CALCnet2.2 progress). Nifty, glad to hear they'll be good command line tools. Have you thought of any subsequent tools beyond a group amanger?
Geopipe
Great start! I'm sure you'll have lots of fun in C programming.

As for things to implement, IF you have the ability and will to do any of them:

Quote:

Some GUI improvements you could implement:

- Asynchronous receiving/sending of files (this includes receiving file lists, etc.)
This would be great, since it's a pain to see it freeze up, especially with TI-Nspires.
This could probably be threading or something like that.
This should also be implemented into the GUI.
Maybe something like this?
http://filezilla-project.org/images/screenshots/fz3_win_main.png (see the bottom part of the window?)

- Transfer queue
This would occur as a result of the above suggestion.
Prioritizing what should be transfered first is a good thing to put in too.
Again, FileZilla should be used as a reference. (Right click the entry at the bottom when the file is still
transferring for an example.)

- Transfer to multiple calcs (could be at the same time, but doesn't have to)
From the above suggestions, this would be great!
Threading would probably be used for this one.

- The folders should be selectable/highlightable.
Clicking on them does not highlight them. It always confuses me as to if it froze or not.
You could also implement an option to make folders transferable to - as in transfer all the compatible
files in the folder to the calc.

- Make a CLI only program without dependencies to GTK/QT.
It's pretty easy for this one, and it would be pretty useful for batch scripts.
I can definitely imagine what could be done with this.
Make it separate so that the configure script doesn't need GTK/QT to compile TiLP. (maybe tilp_cli binary?)

That's all! This program is really nice, and has a lot of potential. Great work!

(directly from http://sourceforge.net/tracker/?func=detail&aid=3018522&group_id=18378&atid=368378)

I could help with the conf script modification (a little bit).

But definitely have fun in the project! Smile Programming doesn't work well without fun. Wink
KermMartian wrote:
Ah, because it's getopt_long stuff, which I was completely missing for some reason. Shock I think I need to have some food or some caffeine (or some CALCnet2.2 progress). Nifty, glad to hear they'll be good command line tools. Have you thought of any subsequent tools beyond a group amanger?


Mini program #2 will be timod which will be for modifying existing files, such as renaming the vars contained in them as well as setting the send to archive flag, basically anything libtifiles2 can do to them I want to have a util to wrap it.

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
That sounds cool. I believe you also mentioned that TiLP can already do a bunch of things from the command line; can you enumerate what those are?
Geopipe

Code:
#include <stdio.h>
#include <stdlib.h>
#include <getopt.h>
#include <tilp2/tifiles.h>

/* Flag set by ‘--verbose’. */
static int verbose_flag;


int
main(int argc, char *argv[])
{
    char c;
    int listmode = 0, extractmode = 0;
    static struct option long_options[] = {
        {"help", 0, 0, 'h'},
        {"version", 0, 0, 'V'},
        {"list", required_argument, 0, 'l'},
        {"extract", required_argument, 0, 'x'},
        {"verbose", 0, &verbose_flag, 'v'},
        {0, 0, 0, 0}
    };
    char *ofile, *ifile;
    int option_index = 0;




    while ((c = getopt_long(argc, argv, ":hVvl:x:", long_options, &option_index)) != -1) {
    switch (c) {
        case 'V':
            printf("This is a test.\n");
            break;
        case 'h':
            printf("This will be help text someday.\n");
            break;
        case 'x':
            extractmode++;
            if (NULL == (ifile = (malloc(strlen(optarg)+1)))) return -1;
            strcpy(ifile,optarg);
            //ifile = optarg;
            break;
        case 'l':
            listmode++;
            ifile = optarg;
            break;
        case ':':       /* -f or -o without operand */
            fprintf(stderr,
                    "Option -%c requires an operand\n", optopt);
            //errflg++;
            break;
        case '?':
            fprintf(stderr,
                    "Unrecognised option: -%c\n", optopt);
            //errflg++;
        }
    }

    if (extractmode){
        int err = 0;
        printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_ungroup_file(ifile, NULL);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_untigroup_file(ifile, NULL);
        else
            fprintf(stderr, "invalid filetype");
    }
    if (listmode){
        int err = 0;
        printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_file_display(ifile);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_file_display_tigroup(ifile);
        else
            fprintf(stderr, "invalid filetype");
    }


    return 0;
}


Updated code that supports listing a files contents, go me. Though I'm not a fan of how much spam tifiles-info spits out, I might have to look into verbosity later.

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
Very nice! You can probably pull my printf() debug statements out of there now, or at least comment them. Smile
Geopipe
TheStorm wrote:

Code:
char *ofile, *ifile;

set these to NULL so you can verify later on whether or not you actually have a file

TheStorm wrote:

Code:
extractmode++;

Huh? You only have 1 input file, you should set extractmode to some constant, this ++ is just... weird

TheStorm wrote:

Code:
if (NULL == (ifile = (malloc(strlen(optarg)+1)))) return -1;

You shouldn't explicitly check against NULL like that, just do an if (!(ifile = malloc(strlen(optarg) + 1))) return -1;
Likewise, don't add unneeded parens - this isn't LISP after all Wink
Also, copying to a new string is a waste of memory (that you aren't freeing later, btw), just store the pointer like you were here:
TheStorm wrote:

Code:
//ifile = optarg;


TheStorm wrote:

Code:
listmode++;

Same as above - wtf is this? Set it to a constant - or even just one. ++ is an increment and store, you aren't incrementing, just setting a flag - so set the dang flag, don't do this weird round-about crap.

TheStorm wrote:

Code:
if (extractmode){
        int err = 0;
        printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_ungroup_file(ifile, NULL);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_untigroup_file(ifile, NULL);
        else
            fprintf(stderr, "invalid filetype");
    }
    if (listmode){
        int err = 0;
        printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_file_display(ifile);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_file_display_tigroup(ifile);
        else
            fprintf(stderr, "invalid filetype");
    }


    return 0;
}



Possible use of undefined ifile - bad bad bad! Hence, initialize to NULL and then verify that it isn't NULL before using it. Also, you can use %p to print the hex value of a pointer - cleaner than %x and casting to (void*). So just printf("%p", ifile) instead of what you're doing.
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
Several notes:
  • making CLI programs (for a subset of the operations, at least) and using these CLI programs from a front-end is usually a good thing, especially under *nix. Mixing everything together, be it TILP / GFM or the non-modular build system of TIGCC Projects, has the downside of very low scriptability.
    That said, having multiple programs instead of once requires extra specification and interfacing work.
  • TILP already has a CLI capability, even if it could be extended (=> spec work and coding work).
  • the TILP binary, even when used in pure CLI mode, depends on GTK+. It could dyamically load the DLL/SO files of the GUI, but as is usual for a technical choice, it has upsides (no GUI dependency for those who don't want it) and downsides (extra complexity).

How many levels of CLI tools do you envision ? I'm not sure we need more than one, because on *nix, shell aliases can turn e.g. "gfm-grp --add" into "gfm-grp-add", and on Windows, people are not into scripting (and maybe PowerShell has aliases, too).
(feel free to come up with better names Very Happy)
Wow, this looks like it has potential, good luck with it

@Kllrnohj - some of that is quick-and-dirty debugging stuff from me from when he was trying to track down a segfault.
Geopipe
KermMartian wrote:
@Kllrnohj - some of that is quick-and-dirty debugging stuff from me from when he was trying to track down a segfault.


Probably due to ifile being allowed to be left undefined - which means it has a garbage value, and attempts to access it will either be met with a segfault or have a horrible outcome.
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
Thanks for the suggestions kllrnohj, I think I have implemented most of them, though extracting tig's is still not working but I'm preatty sure thats a libtifiles2 bug as listing their contents works fine.

I plan on submitting patches to Lionel to make it so most of these routines can accept input from stdin and output to stdout so that my program can, as of now that is not possible so piping will have to wait.
Though if lionel doesn't wish to add that to the libs I can implement it my self fairly easily using the routines libtifiles2 exports.


Code:
#include <stdio.h>
#include <stdlib.h>
#include <getopt.h>
#include <tilp2/tifiles.h>

/* Flag set by ‘--verbose’. */
static int verbose_flag;

//enum modes {MODE_HELP, MODE_EX, MODE_LS,MODE_CREATE, MODE_ADD, MODE_DEL, NULL};


int
main(int argc, char *argv[])
{
    char c;
    enum modes {MODE_NONE, MODE_HELP, MODE_EX, MODE_LS, MODE_CREATE, MODE_ADD, MODE_DEL} curmode;
    curmode=MODE_NONE;
    static struct option long_options[] = {
        {"help", 0, 0, 'h'},
        {"version", 0, 0, 'V'},
        {"list", required_argument, 0, 'l'},
        {"extract", required_argument, 0, 'x'},
        {"verbose", 0, &verbose_flag, 'v'},
        {0, 0, 0, 0}
    };
    char *ofile=NULL, *ifile=NULL;
    int option_index = 0;




    while ((c = getopt_long(argc, argv, ":hVvl:x:", long_options, &option_index)) != -1) {
    switch (c) {
        case 'V':
            printf("Insert Verison Text here\n");
            break;
        default:
        case 'h':
            curmode=MODE_NONE;
            break;
        case 'x':
            if (curmode==MODE_NONE && optarg){
                curmode = MODE_EX;
                //if (ifile = (malloc(strlen(optarg)+1))) return -1;
                //strcpy(ifile,optarg);
                ifile=optarg;
            }
            else {
                curmode = MODE_HELP;
            }
            break;
        case 'l':
            if (curmode==MODE_NONE && optarg){
                curmode = MODE_LS;
                //if (ifile = (malloc(strlen(optarg)+1))) return -1;
                //strcpy(ifile,optarg);
                ifile=optarg;
            }
            else {
                curmode = MODE_NONE;
            };
            ifile = optarg;
            break;
        case ':':       /* -f or -o without operand */
            fprintf(stderr,
                    "Option -%c requires an operand\n", optopt);
            curmode=MODE_HELP;
            break;
        case '?':
            fprintf(stderr,
                    "Unrecognised option: -%c\n", optopt);
            curmode=MODE_HELP;
        case 'v':
            verbose_flag++;
            break;
        }
    }
    if (curmode==MODE_HELP || curmode==MODE_NONE) printf("insert help text here\n");

    if (curmode==MODE_EX){
        int err = 0;
        //printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_ungroup_file(ifile, NULL);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_untigroup_file(ifile, NULL);
        else
            fprintf(stderr, "invalid filetype");
    }
    if (curmode==MODE_LS){
        int err = 0;
        //printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_file_display(ifile);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_file_display_tigroup(ifile);
        else
            fprintf(stderr, "invalid filetype");
    }


    return 0;
}

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
Kllrnohj wrote:
KermMartian wrote:
@Kllrnohj - some of that is quick-and-dirty debugging stuff from me from when he was trying to track down a segfault.


Probably due to ifile being allowed to be left undefined - which means it has a garbage value, and attempts to access it will either be met with a segfault or have a horrible outcome.
Hence my printf statements, but it turned out that Jonimus was putting incorrect values into the option struct.

Jonimus, did you have any thoughts on Lionel's point about levels of tools?
Geopipe
KermMartian wrote:
Kllrnohj wrote:
KermMartian wrote:
@Kllrnohj - some of that is quick-and-dirty debugging stuff from me from when he was trying to track down a segfault.


Probably due to ifile being allowed to be left undefined - which means it has a garbage value, and attempts to access it will either be met with a segfault or have a horrible outcome.
Hence my printf statements, but it turned out that Jonimus was putting incorrect values into the option struct.

Jonimus, did you have any thoughts on Lionel's point about levels of tools?
My thought was two main utils for manipulating the files, one aimed at use with groups and another for non-group files, though I know the fileformat makes the distinction there kinda blurry. We could alias those as much as we want or even make it all one binary if needed. My goal is to keep the code base as simple as possible and anything to complex should be considered a canidate to be added to one of the libs.

As for TiLP I think it would be worth the effort to ship a tilp-cli binary that could be used with either alternative front end UI's or scripts. That would open the door for everything from Makefiles that send the file after building or .net UI's being written for Windows.

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
I like the idea of combining it all into a single binary, since as you say, the TI file format blurs the line between groups and non-groups, and you could enable plenty more code reuse that way, I'm sure. I don't hate the idea of a tilp-cli binary as well, particularly if it could be used to make a build script that would automatically upload the successfully-compiled program to an attached calculator.
Geopipe
KermMartian wrote:
I like the idea of combining it all into a single binary, since as you say, the TI file format blurs the line between groups and non-groups, and you could enable plenty more code reuse that way, I'm sure. I don't hate the idea of a tilp-cli binary as well, particularly if it could be used to make a build script that would automatically upload the successfully-compiled program to an attached calculator.
I would like the idea of combining them as well but it does add a layer of compelxity that I don't care for as much. It would require a lot more planing on my part as to how the interface will work.

As for tilp-cli, tilp does already support sending files without opening the GUI but that is about it. My plan was to basically strip that fuctionality out into a separate program and then slowly add the other functions the that are currently only accessible from the GUI. If I get really motivated I'd kinda like to implement an interactive mode similar in functionality to how sftp or smbclient work.

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
TheStorm wrote:

Code:

    enum modes {MODE_NONE, MODE_HELP, MODE_EX, MODE_LS, MODE_CREATE, MODE_ADD, MODE_DEL} curmode;


What the hell is that?

You had it correct with your enum at the top of the file. Whatever the hell that is, don't do it.

Quote:

Code:
    if (curmode==MODE_HELP || curmode==MODE_NONE) printf("insert help text here\n");


You should probably quit if you don't have something to do. Sure, you won't execute any other code with your ifs, but that can change. It is easier to follow if you simply exit after printing your help.

Quote:

Code:
    if (curmode==MODE_EX){
        int err = 0;
        //printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_ungroup_file(ifile, NULL);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_untigroup_file(ifile, NULL);
        else
            fprintf(stderr, "invalid filetype");
    }
    if (curmode==MODE_LS){
        int err = 0;
        //printf("%x [%s]\n",(void*)ifile,ifile);
        if (tifiles_file_is_group(ifile))
            tifiles_file_display(ifile);
       else if(tifiles_file_is_tigroup(ifile))
            tifiles_file_display_tigroup(ifile);
        else
            fprintf(stderr, "invalid filetype");
    }


    return 0;
}


Still aren't checking if ifile is null Smile
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
TI-Utils: CLI tools for manipulating calc files on your PC
» Goto page 1, 2, 3  Next
» View previous topic :: View next topic  
Page 1 of 3
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement