This project is my longstanding attempt to create a command line utility and Python library for accessing, searching, and downloading projects from ticalc.org (and other sites).

Current version is 1.1; 2.0 has been in development for a while but is hopefully nearing a release with possible support for downloading programs from Cemetech and Omnimaga as well.

Development is generally happening at github- you can clone a snapshot from there. Or, you can download the latest version from ticalc.org for Windows or Linux. (Downloads are also hosted on Cemetech for Windows and Linux). Once it's downloaded, extract the archive and run "python setup.py install". Note that if you're on Linux, this command will probably require root access (su or sudo); no matter what platform you are on you will need Python 2.x to run it.

After installing it, you can run it on the command line using:

python calcpkg.py [command] [filename]

Note that if you're using 2.0 or higher (or the 2.0 snapshots from github), on Linux you should instead be using:
calcpkg [command] [filename]

The first command you run should run is "update"- this will create a local copy of the master.index file from ticalc.org that calcpkg knows how to use for searching and so on. Then you can run "calcpkg search" and "calcpkg install", which will list and download programs. For a full list of all possible options, run "calcpkg -h" (or possibly refer to the README).

I haven't tried this yet, but could you explain what it does a little more? Does it download the entire .zip from ticalc.org?

Yes:


Code:


So in this example, I first list "Wacky Fun Random Numbar Generator", and I am told there is one result and given it's full name and category.

Then I get it, which does the search again but this time downloads the zip file- producing a "generate_os.zip" in the same directory as the script.

Does that make more sense?

Ok. That makes sense, thanks.

Seems like a very cool project, but I could see ticalc.org not being thrilled. Back in the day I had a signature bar generator that would display your ticalc.org author stats live, but that quickly got blocked. There was also a standalone program that would generate detailed statistics about all your files and such, and I think it got 86ed.

Why wont they be thrilled? they will still get downloads and all.

True, but in the past they have been unhappy when a third-party application or webapp starts pulling down a lot of data at once, since their server, while decent enough, was at one point surprisingly ancient for such a popular site. I believe that they have since upgraded, but I would not be surprised if they still had a similar policy. At any rate, I think it might be a positive thing if people with such projects considered contacting nikky, tev, or chronomex to bounce the idea off them, but it's up to what the author feels is necessary, and the author of such programs (not just TC01) would know best if such a tool would use a super-ordinary amount of bandwidth.

It could be less-bandwith heavy I suppose.

The search command uses the websearch.cgi script- which is what the Firefox search plugin for ticalc.org uses. That doesn't produce much bandwith (it's just the equivalent of a search), so the quicklist command is pretty fast.

The get/list commands... because they open all the search results and download all their files... they are slow and therefore maybe bandwith heavy.

A better way to do it would be to use the master index file on ticalc.org, because the index contains both the zip file name, the category, and the name of the file itself. This would make things much faster and lighter for the server, so I really need to rewrite it to work like this.

That sounds like a great idea. That should make ticalc.org's server happier, and it will make your users happier too because searches, as you said, will complete much faster.

Update!

I've completely changed the way this works.

Instead of searching ticalc.org using the websearch, it uses cached package lists (files.index and names.index). These cached files (which are why the download is larger) simply contain parsed forms of the master.index (each line of files.index has a path/filename on ticalc.org, each line of names.index has the name corresponding with the file on the same line in files.index). This means that for searches and counts (I added a "count" command), the script doesn't need the Internet. Searches are now pretty fast.

To download, obviously you need the Internet, but the script can pull the cached paths out of files.index, so no searching is involved.

Then, when you want to update the index files, you use the new "update" command- which parses master.index and recreates the names.index and files.index files. It will take some time to run, though.

The index files must be in the same directory as the script when you run it.

Also, I've uploaded this version to the Cemetech and ticalc.org archives (Omni doesn't have a folder for comp utilities).

The download is here:

http://www.cemetech.net/programs/index.php?mode=file&path=/win/calcpkg.zip

And a screenshot!

Seems fine to me.

Thanks nikky! Hmm, maybe I should resurrect my ticalc.org stats signature one of these days with better caching.

Odd.

I tried running this on my Fedora partition. At first, the output from running "calcpkg.py list" was screwy, but it still worked (it was still able to download packages fine when I ran "calcpkg.py get").

Then I ran an update to rewrite the package index files and the output looked normal.

So all I can say is, run a "calcpkg.py update" before anything else when trying it on Linux.

Eek, that sounds painful. Did you think of any possible causes for this bug?

I suspect it's something to do with the difference between Windows and Unix/Linux line endings.

tev pointed out another issue- the "#!/usr/bin/env python" at the top of the script has its slashes flipped (so it's #!\usr\bin\env python"). I don't know if that is also related (or doing anything other than stopping Linux users from omitting the 'python' before 'calcpkg.py').

Hmm, both good points. What are you using to split lines, regex or split()? And that pound bang line could definitely cause problems.

Wow, fail.

I think what nikky is so subtly trying to say is that among other things, #!\usr\bin\env python has escaped characters u, b, and e, which are all meaningless and might certainly throw the interpreter for a loop.

I've fixed the #!/usr/bin/env python and added case insensitive searching. I think it'll just be best to distribute one copy for Windows and one for Unix/Linux, with the different line endings set.

I still need to get multiple files with the same name working, and a switch for searching in a specified directory. Once that is done, I'll upload a new version.

I also integrated it with an IRC bot... sort of. The bot can do "!calcpkg count text" and say in the channel the number of results. It can do "!calcpkg list text", but since saying it all in the channel would be bad, it pastebins the results and just gives back a link to that Pastebin. And "!calcpkg update" can be ran as well.

The Twisted framework for Python doesn't support sending files to people with dcc, so I can't make it download the files and send them to you.