Don't have an account? Register now to chat, post, use our tools, and much more.
Latest Headlines
Online Users
There are 89 users online: 2 members, 65 guests and 22 bots.
Members: lafferjm.
Bots: Spinn3r (1), MSN/Bing (1), VoilaBot (1), Googlebot (18), MSN/Bing (1).
RSS & Social Media
SAX
You must log in to view the SAX chat widget
|
| Author |
Message |
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
 Posted: 24 Nov 2010 09:07:12 pm Post subject: ticalc.org "Package Manager" |
 |
|
This project is my longstanding attempt to create a command line utility and Python library for accessing, searching, and downloading projects from ticalc.org (and other sites).
Current version is 1.1; 2.0 has been in development for a while but is hopefully nearing a release with possible support for downloading programs from Cemetech and Omnimaga as well.
Development is generally happening at github- you can clone a snapshot from there. Or, you can download the latest version from ticalc.org for Windows or Linux. (Downloads are also hosted on Cemetech for Windows and Linux). Once it's downloaded, extract the archive and run "python setup.py install". Note that if you're on Linux, this command will probably require root access (su or sudo); no matter what platform you are on you will need Python 2.x to run it.
After installing it, you can run it on the command line using:
python calcpkg.py [command] [filename]
Note that if you're using 2.0 or higher (or the 2.0 snapshots from github), on Linux you should instead be using:
calcpkg [command] [filename]
The first command you run should run is "update"- this will create a local copy of the master.index file from ticalc.org that calcpkg knows how to use for searching and so on. Then you can run "calcpkg search" and "calcpkg install", which will list and download programs. For a full list of all possible options, run "calcpkg -h" (or possibly refer to the README).
_________________
Solar89: A 68k Dev Tool for z80 Basic
renamr68k: Utility script for 68k development
Package manager for ticalc.org (command line tool to access it)
Memory Usage: a tool for the 89 to show your memory usage
Last edited by TC01 on 20 Nov 2012 09:47:35 pm; edited 2 times in total |
|
| Back to top |
|
|
souvik1997
Guru-in-Training
Joined: 19 Apr 2010
Posts: 2870
|
| Posted: 25 Nov 2010 01:25:56 am Post subject: |
|
|
I haven't tried this yet, but could you explain what it does a little more? Does it download the entire .zip from ticalc.org?
_________________
CALCnet Tournament-38%
deviantArt
 |
|
| Back to top |
|
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
| Posted: 25 Nov 2010 10:39:40 am Post subject: |
|
|
| souvik1997 wrote: | | I haven't tried this yet, but could you explain what it does a little more? Does it download the entire .zip from ticalc.org? |
Yes:
Code:
C:\Python26\Dev\calcpkg>python calcpkg.py list "Wacky Fun Random Numbar Generator Operating System"
Search for 'Wacky Fun Random Numbar Generator Operating System' returned 1 result(s)
File Name Category (excluded in quick search)
=========================================================================================================================
Wacky Fun Random Numbar Generator Operating System DOS Utilities
C:\Python26\Dev\calcpkg>python calcpkg.py get "Wacky Fun Random Numbar Generator Operating System"
Search for 'Wacky Fun Random Numbar Generator Operating System' returned 1 result(s)
File Name Category (excluded in quick search)
=========================================================================================================================
Wacky Fun Random Numbar Generator Operating System DOS Utilities
Are you sure you want to download these files [Y/N]? y
Downloading Wacky Fun Random Numbar Generator Operating System from http://www.ticalc.org/pub/dos/generate_os.zip
Download complete!
C:\Python26\Dev\calcpkg>
So in this example, I first list "Wacky Fun Random Numbar Generator", and I am told there is one result and given it's full name and category.
Then I get it, which does the search again but this time downloads the zip file- producing a "generate_os.zip" in the same directory as the script.
Does that make more sense?
_________________
Solar89: A 68k Dev Tool for z80 Basic
renamr68k: Utility script for 68k development
Package manager for ticalc.org (command line tool to access it)
Memory Usage: a tool for the 89 to show your memory usage |
|
| Back to top |
|
|
souvik1997
Guru-in-Training
Joined: 19 Apr 2010
Posts: 2870
|
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 26 Nov 2010 12:46:38 am Post subject: |
|
|
Seems like a very cool project, but I could see ticalc.org not being thrilled.  Back in the day I had a signature bar generator that would display your ticalc.org author stats live, but that quickly got blocked.  There was also a standalone program that would generate detailed statistics about all your files and such, and I think it got 86ed.
_________________
 
 
 |
|
| Back to top |
|
|
Eeems
Super-Expert
Joined: 22 Jan 2009
Posts: 832
Location: Edmonton, Alberta
|
| Posted: 26 Nov 2010 06:10:36 pm Post subject: |
|
|
Why wont they be thrilled? they will still get downloads and all.
_________________
        |
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 26 Nov 2010 07:43:39 pm Post subject: |
|
|
| Eeems wrote: | | Why wont they be thrilled? they will still get downloads and all. |
True, but in the past they have been unhappy when a third-party application or webapp starts pulling down a lot of data at once, since their server, while decent enough, was at one point surprisingly ancient for such a popular site. I believe that they have since upgraded, but I would not be surprised if they still had a similar policy. At any rate, I think it might be a positive thing if people with such projects considered contacting nikky, tev, or chronomex to bounce the idea off them, but it's up to what the author feels is necessary, and the author of such programs (not just TC01) would know best if such a tool would use a super-ordinary amount of bandwidth.
_________________
|
|
| Back to top |
|
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 28 Nov 2010 02:39:30 pm Post subject: |
|
|
That sounds like a great idea. That should make ticalc.org's server happier, and it will make your users happier too because searches, as you said, will complete much faster.
_________________
|
|
| Back to top |
|
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
| Posted: 28 Nov 2010 05:31:59 pm Post subject: |
|
|
Update!
I've completely changed the way this works.
Instead of searching ticalc.org using the websearch, it uses cached package lists (files.index and names.index). These cached files (which are why the download is larger) simply contain parsed forms of the master.index (each line of files.index has a path/filename on ticalc.org, each line of names.index has the name corresponding with the file on the same line in files.index). This means that for searches and counts (I added a "count" command), the script doesn't need the Internet. Searches are now pretty fast.
To download, obviously you need the Internet, but the script can pull the cached paths out of files.index, so no searching is involved.
Then, when you want to update the index files, you use the new "update" command- which parses master.index and recreates the names.index and files.index files. It will take some time to run, though.
The index files must be in the same directory as the script when you run it.
Also, I've uploaded this version to the Cemetech and ticalc.org archives (Omni doesn't have a folder for comp utilities).
_________________
Solar89: A 68k Dev Tool for z80 Basic
renamr68k: Utility script for 68k development
Package manager for ticalc.org (command line tool to access it)
Memory Usage: a tool for the 89 to show your memory usage |
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
|
| Back to top |
|
|
allynfolksjr
Minor Calculator Deity
Joined: 20 Apr 2005
Posts: 1273
|
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 28 Nov 2010 08:54:59 pm Post subject: |
|
|
| allynfolksjr wrote: | | Seems fine to me. |
Thanks nikky! Hmm, maybe I should resurrect my ticalc.org stats signature one of these days with better caching.
_________________
|
|
| Back to top |
|
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 29 Nov 2010 11:08:41 pm Post subject: |
|
|
Eek, that sounds painful. Did you think of any possible causes for this bug?
_________________
|
|
| Back to top |
|
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 30 Nov 2010 10:17:34 am Post subject: |
|
|
Hmm, both good points. What are you using to split lines, regex or split()? And that pound bang line could definitely cause problems.
_________________
|
|
| Back to top |
|
|
allynfolksjr
Minor Calculator Deity
Joined: 20 Apr 2005
Posts: 1273
|
|
| Back to top |
|
|
KermMartian
Site Admin
Joined: 14 Mar 2005
Posts: 55773
Location: Earth, Sol, Milky Way
|
| Posted: 30 Nov 2010 11:43:58 am Post subject: |
|
|
| allynfolksjr wrote: | | Wow, fail. |
I think what nikky is so subtly trying to say is that among other things, #!\usr\bin\env python has escaped characters u, b, and e, which are all meaningless and might certainly throw the interpreter for a loop.
_________________
|
|
| Back to top |
|
|
TC01
Member
Joined: 28 Jul 2010
Posts: 166
|
| Posted: 30 Nov 2010 04:38:14 pm Post subject: |
|
|
I've fixed the #!/usr/bin/env python and added case insensitive searching. I think it'll just be best to distribute one copy for Windows and one for Unix/Linux, with the different line endings set.
I still need to get multiple files with the same name working, and a switch for searching in a specified directory. Once that is done, I'll upload a new version.
I also integrated it with an IRC bot... sort of. The bot can do "!calcpkg count text" and say in the channel the number of results. It can do "!calcpkg list text", but since saying it all in the channel would be bad, it pastebins the results and just gives back a link to that Pastebin. And "!calcpkg update" can be ran as well.
The Twisted framework for Python doesn't support sending files to people with dcc, so I can't make it download the files and send them to you.
_________________
Solar89: A 68k Dev Tool for z80 Basic
renamr68k: Utility script for 68k development
Package manager for ticalc.org (command line tool to access it)
Memory Usage: a tool for the 89 to show your memory usage |
|
| Back to top |
|
|
|
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.
» Go to Registration page
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
© Copyright 2000-2013 Cemetech & Kerm Martian :: Page Execution Time: 0.043939 seconds.
|
![[t]](/forum/templates/Cemetech6/images/twitter.png)
![[t]](/forum/templates/Cemetech6/images/fb.png)
![[RSS]](/forum/templates/Cemetech6/images/rss.png)
