Very nice updates. It's processed through the Cemetech file upload queue now, by the way.

So, I finally remembered that tev suggested parsing the file's category page to get the info page for a file.

So, this morning, I wrote code to do just that, modelling the output on yum's info command. Here's some sample output. Ignore the weird coloring- I didn't feel like making a screenshot, so I copy-pasted my terminal output into a code tag.


Code:


As you can see, "info" takes all the same arguments as get/install and search/list.

The one thing I haven't done is make it handle multiple authors (which is why I'm not releasing right now). And because it does HTML parsing (whereas most of the other commands use the local index files), if you ask for info on multiple files at once it will be slow.

Also, before, the file's line encoding was Windows, which meant that you couldn't run './calcpkg.py', you needed to run 'python calcpkg.py'. Now, the output is Unix and both invocations work fine.

About nine hours later... I've released version 1.0!

It's available from ticalc.org here for Windows and here for Linux.

As part of this release, I completely restructured the program. Before, it was a little clunky- there was one Python script, you had to have it in the same directory as the index files and where things were downloaded to. After doing the above, I rewrote the script so there was a script (calcpkg) and a Python package (ticalcDotOrg).

So what does that mean? Well, for one, you don't need to worry about where the index files are anymore. And the system now automatically downloads to your home downloads folder- so on Windows that would be C:\Users\username\Downloads, and on Linux that's /home/username/Downloads.

It also means you don't need to type "python calcpkg.py" anymore, you can just type "calcpkg.py". (On Linux, I removed the ".py" suffix, so you didn't have to type "calcpkg.py" and could just type "calcpkg". However, Windows apparently didn't like that, so for Windows it's still called calcpkg.py.)

And I did get the install command to automatically extract tar-compressed files too.

To install calcpkg, you have a large number of options. On Windows, you can run either of the executables (one is for win32, one is for amd-win64, or 64-bit), or you can extract the zip archive and run "python setup.py install". On Linux, I've provided an RPM package for Fedora, OpenSUSE, Mandriva, etc. users. Python can actually automatically make rpm packages, which is a nice feature (especially since I use Fedora ). Unfortunately, there is no such support for making .deb packages (at least, not in the official distutils). So users of other Linux systems will have to extract the .tar.gz file and run "python setup.py install".

I've uploaded 1.1 to ticalc.org and Cemetech archives. This is just a minor bugfix update; I fixed the clean command and the logging system (both of which I inadvertently broke with the 1.0 update/rewrite). Clean now just deletes the index files (in case they become corrupted or something). As for logging, it is enabled by default in the Linux version (and logs to /tmp/calcpkg.log). It's disabled for now under Windows until I figure out what exactly I want to do with it there.

And I combined the -n and -p options to one global -y, --assume-yes option (having two opt switches that did the same thing was silly).

The main point of this release, though, is a couple of fixes to the way the -f switch (search by archive file name rather than title) works. And that's because I wrote some Python code that uses calcpkg for Kerm's Sandpaper FTP gCn project, but it requires the fix to work properly.

It's been ages since I've posted about this project... I've been working on it recently, so I thought I'd bump this.

First off, there is now a github repository for this project, so you can see I actually have been working on it lately.

I'm vaguely approaching the milestone for a 2.0 release. Aside from vastly cleaned up and improved code (the only thing that has yet to be rewritten from the old 0.5/0.8 days is the actual index searching code, which someday probably should be), the big feature is support for multiple repositories.

Unfortunately I don't actually have any multiple repositories to show off quite yet. There are templates for cemetech and omnimaga under various stages of completion (and it's trivial to add more, they are just python files that get dropped into a directory), but for various reasons neither are quite yet acceptable. I'd love to have at least one of these working before I release 2.0 (yes, I actually still release projects!), but if that proves impractical I may just create a test repository of some sort to show off the feature.

But anyways... here's the current status of both- suggestions or whatnot are welcome:

The Cemetech one works pretty well, I just don't have an easy way of getting a list of all files (like this). I guess I could make an awkward hack of some sort that crawls through the archives, but I imagine it'd work a lot nicer if a script or something could be run off the Cemetech servers directly (again, like ticalc.org).

The Omnimaga one is a bit messier, because Omnimaga uses file IDs in file URLs rather than paths, the way Cemetech and ticalc.org do. To be more specific: a ticalc.org download URL points at /83plus/asm/games/filename.zip (or something), whereas an Omni download URL points at id=350 (or something). On the plus side, this makes it easier to get a list of all files- I simply start at file "1" and loop until I've gotten all n files, where n is the number of files in the archives (a statistic that's available elsewhere on the site). But on the other... it would break searching directly by "file name"- because the data that would need to be stored as "file name" would be file IDs (in other words, just a number), rather than something like "filename.zip".

Double post, technically, but it has been more than a year... whoops.

After finally growing frustrated enough at the lack of much progress on this project, I decided to do try and actually make progress now that I actually have free time and am feeling productive (but don't wish to do anything academic in nature).

I've created a new branch on Github with working Cemetech support.

The list of files is fetched by recursively stepping through file directory pages and making a list of all the files within. I was uneasy about doing this (the main reason I had held off on adding such support) because I would think it a bit more bandwidth intensive than just downloading an index of all such files. And plus, I was uncertain if a master index of files would ever actually be added to Cemetech.

Ironically enough, it's actually much faster (on my end) than downloading the master index off of ticalc.org, probably because that thing is huge and opening a few dozen web pages takes almost no time at all.

Kerm: if doing this spidering through file categories on an update is acceptable, after I polish up a few more things, I'd like to merge this branch in and release an official version 2.0.

*resurrects this thread yet again*

After growing impatient with myself, I've finally released calcpkg v2.0, three years after the last release to ticalc.org and Cemetech.

If you've read above, you're probably aware of some of the changes that have been sitting on github for quite a while, but to summarize...

-Calcpkg is now installable through pip! "pip install calcpkg" is probably the fastest way to get it, though I've updated the ticalc.org and Cemetech archives with the new release.
As a consequence of this, setuptools is now a dependency. Sorry.

-Support for Cemetech and Omnimaga, with an extensible system enabling more community site archives to be added! TI-Planet support is planned in the hopefully-not-three-years-away future.

-An overhaul of command line options and further modularization / code cleanup / etc of bits of the project. (At least, that's what my changelog claims, it's been so long...).

-Changes made by KermM for Sandpaper to the calcrepo backend were merged in as well.

(For a more detailed changelog, take a look at the git commit history on github).

If you forget what this thing is: it's a command line tool to query ticalc.org (and now other sites). As a quick and silly example, "calcpkg -r ticalc list Quadratic" will probably list all the correctly-spelled quadratic solvers on ticalc.org, and "calcpkg -r ticalc get Quadratic" would download them all (if, for some reason, you really wanted all of them).

The backend library is also available for anyone who wants to do something that requires having access to the {ticalc.org, Cemetech, Omnimaga} archives; as I mentioned above, components of calcpkg were used as part of Sandpaper.

If you have a feature request / bug report, either tell me about it here or make use of the issue tracker.

Also, the screenshots on the project pages are now out of date, so I should really update them...

Good to see you could find some time to make progress and make an official release

Oh, you know, you reminded me about an official index of all files in the Cemetech Archives, and I still failed to implement that. Hopefully I'll find the time to throw that together sometime in the very near future indeed. Congratulations on this new update and the features that it includes, TC01. Do you have any new stuff that you plan to add after this?

Found and fixed a few bugs that have been sitting in git for a few days, so I'll be pushing v2.1 (with those fixes) to PyPI, ticalc.org, and Cemetech shortly.



No rush, I understand that you're very busy.

I have a few plans: the most interesting, I think, was Eeems' suggestion that I add the ability to upload files to archives instead of just downloading them; i.e. something like "calcpkg upload project.zip" would upload the file. The exact details still need to be worked out, though.

Also, on IRC, you mentioned linking the two files- if you want to do that, I think I'd prefer to have the Windows version linked to the Linux version, since *nix is better at zip than Windows is at tar.

(There's really no difference, other than which archive format I uploaded)