Author Topic: ticalc.org "Package Manager"  (Read 52516 times)

0 Members and 3 Guests are viewing this topic.

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: ticalc.org "Package Manager"
« Reply #60 on: August 26, 2011, 09:44:28 pm »
That is awesome TC01 :D

Offline Eeems

  • Mr. Dictator
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6267
  • Rating: +318/-36
  • little oof
    • View Profile
    • Eeems
Re: ticalc.org "Package Manager"
« Reply #61 on: August 26, 2011, 09:44:58 pm »
If you want I can setup a package for pacman and host it on my repo. That way all the arch users who use my repo can install it with pacman -S calcpkg :)
/e

Offline TC01

  • LV6 Super Member (Next: 500)
  • ******
  • Posts: 344
  • Rating: +9/-0
    • View Profile
Re: ticalc.org "Package Manager"
« Reply #62 on: August 26, 2011, 09:59:03 pm »
If you want I can setup a package for pacman and host it on my repo. That way all the arch users who use my repo can install it with pacman -S calcpkg :)

That'd be great, Eeems. :)



The userbars in my sig are links embedded links.

And in addition to calculator (and Python!) stuff, I mod Civilization 4 (frequently with Python).

Offline Eeems

  • Mr. Dictator
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6267
  • Rating: +318/-36
  • little oof
    • View Profile
    • Eeems
Re: ticalc.org "Package Manager"
« Reply #63 on: August 27, 2011, 12:44:57 am »
If you want I can setup a package for pacman and host it on my repo. That way all the arch users who use my repo can install it with pacman -S calcpkg :)

That'd be great, Eeems. :)
Alright I get working on it then

EDIT: Ok done, I think. Anybody who uses arch and wants it, feel free to add my database to /etc/pacman.conf
Code: [Select]
[omni]
Server = http://withg.us.to/eeems/$repo
Server = http://repo.julosoft.net/$repo/os/$arch
then run
Code: [Select]
pacman -Sy calcpkgto install it
« Last Edit: August 27, 2011, 02:10:45 am by Eeems »
/e

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: ticalc.org "Package Manager"
« Reply #64 on: August 27, 2011, 02:18:19 am »
Nice stuff Eeems :D

Offline TC01

  • LV6 Super Member (Next: 500)
  • ******
  • Posts: 344
  • Rating: +9/-0
    • View Profile
Re: ticalc.org "Package Manager"
« Reply #65 on: September 02, 2011, 01:45:14 pm »
I've uploaded 1.1 to ticalc.org and Cemetech archives. This is just a minor bugfix update; I fixed the clean command and the logging system (both of which I inadvertently broke with the 1.0 update/rewrite). Clean now just deletes the index files (in case they become corrupted or something). As for logging, it is enabled by default in the Linux version (and logs to /tmp/calcpkg.log). It's disabled for now under Windows until I figure out what exactly I want to do with it there.

And I combined the -n and -p options to one global -y, --assume-yes option (having two opt switches that did the same thing was silly).

The main point of this release, though, is a couple of fixes to the way the -f switch (search by archive file name rather than title) works. And that's because I wrote some Python code that uses calcpkg for Kerm's Sandpaper FTP gCn project, but it requires the fix to work properly.



The userbars in my sig are links embedded links.

And in addition to calculator (and Python!) stuff, I mod Civilization 4 (frequently with Python).

Offline Eeems

  • Mr. Dictator
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6267
  • Rating: +318/-36
  • little oof
    • View Profile
    • Eeems
Re: ticalc.org "Package Manager"
« Reply #66 on: September 02, 2011, 04:21:26 pm »
downloading and updating my hosted package now
/e

Offline TC01

  • LV6 Super Member (Next: 500)
  • ******
  • Posts: 344
  • Rating: +9/-0
    • View Profile
Re: ticalc.org "Package Manager"
« Reply #67 on: November 20, 2012, 09:34:34 pm »
It's been ages since I've posted about this project (or indeed at all)... I've been working on it recently, so I thought I'd bump this. (And I probably should update the first post while I'm at it- doing that now).

First off, there is now a github repository for this project, so you can see I actually have been working on it lately. ;)

I'm vaguely approaching the milestone for a 2.0 release. Aside from vastly cleaned up and improved code (the only thing that has yet to be rewritten from the old 0.5/0.8 days is the actual index searching code, which someday probably should be), the big feature is support for multiple repositories.

Unfortunately I don't actually have any multiple repositories to show off quite yet. There are templates for cemetech and omnimaga under various stages of completion (and it's trivial to add more, they are just python files that get dropped into a directory), but for various reasons neither are quite yet acceptable. I'd love to have at least one of these working before I release 2.0 (yes, I actually still release projects!), but if that proves impractical I may just create a test repository of some sort to show off the feature.

But anyways... here's the current status of both- suggestions or whatnot are welcome:

The Cemetech one works pretty well, I just don't have an easy way of getting a list of all files (like this). I guess I could make an awkward hack of some sort that crawls through the archives, but I imagine it'd work a lot nicer if a script or something could be run off the Cemetech servers directly (again, like ticalc.org).

The Omnimaga one is a bit messier, because Omnimaga uses file IDs in file URLs rather than paths, the way Cemetech and ticalc.org do. To be more specific: a ticalc.org download URL points at /83plus/asm/games/filename.zip (or something), whereas an Omni download URL points at id=350 (or something). On the plus side, this makes it easier to get a list of all files- I simply start at file "1" and loop until I've gotten all n files, where n is the number of files in the archives (a statistic that's available elsewhere on the site). But on the other... it would break searching directly by "file name"- because the data that would need to be stored as "file name" would be file IDs (in other words, just a number), rather than something like "filename.zip".



The userbars in my sig are links embedded links.

And in addition to calculator (and Python!) stuff, I mod Civilization 4 (frequently with Python).

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: ticalc.org "Package Manager"
« Reply #68 on: November 20, 2012, 09:58:23 pm »
Good to hear this is progressing again. :)

By the way, is getting ticalc.org files tricky when it comes to hotlinking? I remember that when someone posted a direct zip file link, it redirected to the directory listing or something.

Also glad you're getting Omni and Cemetech too :D. You should do TI-Planet as well :D (although in their case, like Omnimaga, a bunch of the files are links to a different server, such as TI OSes, rather than an uploaded file)

Offline Juju

  • Incredibly sexy mare
  • Coder Of Tomorrow
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 5730
  • Rating: +500/-19
  • Weird programmer
    • View Profile
    • juju2143's shed
Re: ticalc.org "Package Manager"
« Reply #69 on: November 20, 2012, 09:59:29 pm »
Hey welcome back ^_^

Actually, you can get the filename from the HTTP header.

(woo 4000th post :3)

Remember the day the walrus started to fly...

I finally cleared my sig after 4 years you're happy now?
THEGAME
This signature is ridiculously large you've been warned.

The cute mare that used to be in my avatar is Yuki Kagayaki, you can follow her on Facebook and Tumblr.

Offline TC01

  • LV6 Super Member (Next: 500)
  • ******
  • Posts: 344
  • Rating: +9/-0
    • View Profile
Re: ticalc.org "Package Manager"
« Reply #70 on: November 20, 2012, 10:31:01 pm »
Good to hear this is progressing again. :)

By the way, is getting ticalc.org files tricky when it comes to hotlinking? I remember that when someone posted a direct zip file link, it redirected to the directory listing or something.

Also glad you're getting Omni and Cemetech too :D. You should do TI-Planet as well :D (although in their case, like Omnimaga, a bunch of the files are links to a different server, such as TI OSes, rather than an uploaded file)

Hmm... I've never had issues with it. My code to download just gets a link to the direct zip and downloads said zip.

I'll add TI-Planet to the list of "things I'll hope to get working"- are there any other calc sites these days that still have sizeable download archives that are worth trying to support?

Hey welcome back ^_^

Actually, you can get the filename from the HTTP header.

(woo 4000th post :3)

I was in the middle of writing out a reply to this and then I realized I may misunderstood- when I click on the "download URL" button, the HTTP header will tell me what the file is that this link actually points at, instead of the "sa=downfile;id=number;" link?

In that case, then, it'll be much easier. :D I could use the "loop through all files" process to figure out the IDs, get the name they were uploaded under from their info page, and then do that to find out what the actual archive filename is. Still more complex than ticalc.org, but at least it'll work.
 
Mind you- it might be better to not do this, especially if this would increment the download counter each time I run it.... maybe it would be easier to just rewrite the search code and create a third index file that will store file IDs or something.



The userbars in my sig are links embedded links.

And in addition to calculator (and Python!) stuff, I mod Civilization 4 (frequently with Python).

Offline Eeems

  • Mr. Dictator
  • Administrator
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 6267
  • Rating: +318/-36
  • little oof
    • View Profile
    • Eeems
Re: ticalc.org "Package Manager"
« Reply #71 on: November 20, 2012, 10:39:53 pm »
I should probably reboot into linux and rebuild the omni repo to include a more recent snapshot of this :P
http://omni.eeems.ca/ <- I've been keeping it semi up to date.
/e

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: ticalc.org "Package Manager"
« Reply #72 on: November 21, 2012, 12:40:17 am »
There are also the following sites that are quite large in amounts of downloads, but since they died years ago, I'm unsure for how long they will remain online, so idk if it's worth it:

ti-bank.fr (still active. Over 1000 files)
planet-casio.com/Fr/ (still active. 1798 files)
casiopeia.net (still active. 84 files)
casiocalc.org (still active, but most download links are broken. Over 300 files)
casiokingdom.org (probably still active, 527 downloads)
calc.org (died in 2006. 2138 files)
calcg.org (died in 2010. Estimated 1000 files)
www.ti83plus.online.fr (died in 2009. 1300+ files)
ti83.free.fr (died in 2000. About 1000 files)

Hopefully the United-TI archives are brought back up one day.

Offline TravisE

  • LV4 Regular (Next: 200)
  • ****
  • Posts: 182
  • Rating: +33/-0
    • View Profile
    • ticalc.org
Re: ticalc.org "Package Manager"
« Reply #73 on: November 21, 2012, 12:48:19 am »
Good to hear this is progressing again. :)

By the way, is getting ticalc.org files tricky when it comes to hotlinking? I remember that when someone posted a direct zip file link, it redirected to the directory listing or something.

Also glad you're getting Omni and Cemetech too :D. You should do TI-Planet as well :D (although in their case, like Omnimaga, a bunch of the files are links to a different server, such as TI OSes, rather than an uploaded file)
Hmm... I've never had issues with it. My code to download just gets a link to the direct zip and downloads said zip.

There is an “issue” with wget, because the server is set up to reject it for some reason. Which is dumb, because it can be trivially bypassed by using the option -U “any-non-wget-user-agent-string”. I think there is a hotlinking protection for images that goes by referrer headers, but I haven't checked to see if it's still active today.
« Last Edit: November 21, 2012, 12:49:19 am by TravisE »
ticalc.org staff member—http://www.ticalc.org/

Offline TC01

  • LV6 Super Member (Next: 500)
  • ******
  • Posts: 344
  • Rating: +9/-0
    • View Profile
Re: ticalc.org "Package Manager"
« Reply #74 on: November 21, 2012, 11:20:00 am »
There is an “issue” with wget, because the server is set up to reject it for some reason. Which is dumb, because it can be trivially bypassed by using the option -U “any-non-wget-user-agent-string”. I think there is a hotlinking protection for images that goes by referrer headers, but I haven't checked to see if it's still active today.

Oh, then I guess this is working because the useragent used by Python's urlllib doesn't get rejected.

Is there a reason why wget is set to be rejected, or is it just some old configuration setting that hasn't been changed for years?



The userbars in my sig are links embedded links.

And in addition to calculator (and Python!) stuff, I mod Civilization 4 (frequently with Python).