TC01, I would be extremely interested in integrating some of that functionality in saxjax for #cemetech. Would you be able to share some of that code with me?

Certainly.

Attached (well, linked here) is the current version of the package manager (adapted slightly for IRC) and the bot. (The bot imports the package manager).

It uses the Twisted framework for Python 2.x- which requires both Twisted and Zope.Interface to run.

It is set to join #cemetech and #omnimaga by default on whatever IRC network you specify when you run it- you specify the network by:

"python calcpkgbot.py #otherchan irc.efnet.net"

#otherchan could be anything- making it #cemetech or #omnimaga would make it only join the two default channels.

There are five commands in there- in addition to !calcpkg list, !calcpkg count, !calcpkg update, and !calcpkg help, I added a !nick to change the bot's nickname remotely (the default is set to "Hex", a Terry Pratchett reference and most likely a nick already in use somewhere).

The update command can only be ran by operators because it's a time/resource intensive operation.

If you send commands to a private message, it will return them to you via private message- so if it's spamming up the channel you can make people take it to privmsg (or just silence it). DJ suggested flood control of some sort- that is the closest I came to doing it now.

I wonder how many quadratic solvers, Qarduatic Slovar, qadatik solvar, qadratique solveur and qradatic sovlers will we be able to find with this program... I bet over 1000 or 2000...

Haha, you think that few? I'm equally curious to give that a try, personally. TC01, I will try to get a chance to test integrating this with saxjax later today and tell you how it goes. You frequent #cemetech though, so I suppose you'll see for yourself.

Getting ready to release an update.

In addition to case insensitive searching and the #! fix, I've:

1. Changed the way downloaded files are named- they contain the path (so "83plus-asm-games-generate.zip" rather than just "generate.zip") to fix the issue with multiple files containing the same name.

2. Added command-line opts for searching. The opts are:

-c [name]: --category=[name]: some category to search in. This can be high level like "83plus" or "win", and it can also be "games" or "math" or "basic" or "os" or whatever.
-g: --game: Searches for games only ("-c games")
-m: --math: Searches for math and science only ("-c math" and "-c games")

These switches only work with the list and get commands. I also added a switch for update, -s or --silent, which keeps output to a minimum. All switches can be viewed with "calcpkg.py -h".

I'm also just going to have a separate Unix and Windows version on ticalc.org (and Cemetech) with the different line endings.

I just need to rewrite the readme, boot into Windows and test the Windows version, and then I'll release.

I have accepted the two versions in the archives. For the sake of your (and my, and my webserver's) sanity, what would your thoughts be on a single archive in both the /win/ and /nix/ folders containing the two executables but only a single copy of the 1.5MB index file?

The bot idea is pretty interesting. I recommend the flood control addition for sure, though, since I am pretty sure there would be abuse otherwise.

Argh, and I still forgot to give that a try. :/ One of these days I swear I'll remember.

That would work, I guess, the only problem is that the index files need to be rebuilt specifically for Linux if they were Windows, and probably for Windows if they were Linux.

So you could still get along without having to run an update first thing, basically.

But I could do it if it would save your webserver some craziness.

Well, ignoring my webserver for a moment, why to the index files need to be different? There's no reason that that should be the case. Is it due to how you split lines in your program? If so, we should make you a line-splitting algorithm that doesn't care about line endings.

Well, it seems that they don't have to be different.

The Windows files didn't seem to work completely when I tried them on Linux before, but the Linux ones worked fine on Windows.

So, they don't have to. I'll upload one version of the file containing both scripts, then?

We should figure out what's not working completely first, though. First of all, how are the two files being created that would make them different? You download them from ticalc.org's webserver, right? Or is your index different from the ticalc.org master index?

Well, here's my "updateTicalcPackageLists" function. fileindex and nameindex are globals with the names of the index files produced.

What it does is read in the ticalc.org master index, delete the old indexes, then go line by line through the read in data and write the name to names.index and the path/archive file name to files.index.

Most likely, I should be using another method to cycle through the lines than slicing.... like "for line in file" or something.


Code:

So, it seems to me that this is our main problem:


Code:
What is your corresponding code to read/split names and files later?

An even longer function, so I'll pastebin it.

Here.

Ah, and there you do "for line in namefile:". According to the relevant page in the Python documentation, Python tries to change line endings everywhere to \n. Can you describe what exactly goes wrong on the Linux version when they share the same index file?

It's the structureSearchOutput function that failed. Instead of printing the URL on one side, then the name after some spacing, it just printed the name in the middle.

At least, it did when I last tried it... I've booted into Windows now though.

In other words "for datum in data:" returns both items in datum[0] instead of the first in datum[0] and the second in datum[1]? One thing I happen to notice, by the way - math is a reserved word, so you might want to be careful about using it.

I've been working on a new version lately. Here are some of the changes:

A list switch (-f) which lets you search for the file name rather than the program name. For instance, if you want to search for all ticalc.org files containing "quadratic" in the filename rather than the name of the program, you'd use -f with list.

I've been typing "search" instead of "list", so I've redirected the "search" command to "list"- both calcpkg.py search and calcpkg.py list will do the same things.

The searching options can now be ran with calcpkg.py get as well.

I also added a logging system, so if you run it with the -l switch every text message printed to the terminal will also be written to calcpkg.log (with a note indicating the date ran).

I added a "clean" command that deletes all .tar.gz and .zip files from the folder that calcpkg.py is in (it also removes the calcpkg.log file). Please note: the files are not sent to the recycle bin/trash, so they're deleted for good.

And finally, I added options to disable the confirmation prompts when running calcpkg.py get (-n) and calcpkg.py update (-p). I'd have preferred them to be the same letter but optparse wouldn't let me.

For people who wish to use calcpkg as part of another program (*cough* making a GUI *cough*), there are several other changes.

The options are now parsed in main(), and then passed as boolean or string arguments to various functions (unlike before, when the opts object was passed directly to each function using it). This will make it easier for another program to search using options.

Finally, I rewrote the text output system. An end user shouldn't notice any changes, but you will if you look at the code. (This is a bit complex). Python will let you redirect print to any object with a write() function. I've done the following:

1. Changed (almost) all instances of "print" to "printcalcpkg()", a custom-defined function.
2. printcalcpkg() will print it's text to whatever object is stored in "calcpkg.output" (the global variable output from inside the program)
3. Set the default output to a "calcpkgOutput" object I've included- which prints to the terminal and has an option to also log output to a logfile (how logging works)

If you want to see an example of how to make a custom "printing" object look at the code.

What does this mean? It means that you could redirect all the print statements to some kind of GUI- a big textbox or something.

This sounds great, thanks for letting us know! I've caught quite a few GUI hints in there; is this something that you're planning? Or are you hoping someone else will decide to make the GUI for it?