Internet Archive Bulk cli Uploading

[manyroads]

Hello all.

I am attempting to bulk upload a bunch of images (pdf files 18GB worth) to the Internet Archive. I think I am almost there but my brain is officially in overload. Maybe a smart person can help. What follows is what I have achieved thus far. I think for an old guy, it's "almost adequate" :lipsrsealed: ... but not quite good "enough".

Here's what I've done & where I'm stuck.

To get the internetarchive (ia) command environment functioning you need to"

Code: Select all

$ sudo apt-get install python-pip python-pip-whl pypy-six
$ pip install --upgrade setuptools
$ sudo pip install internetarchive

Next the InternetArchive Command line interface needs to be installed and verified:

Code: Select all

$ curl -LOs https://archive.org/download/ia-pex/ia
$ chmod +x ia
$ ./ia help

The remaining instructions may be found here: https://archive.org/services/docs/api/i ... tml#upload

The challenge I'm having is in building a functioning csv spreadsheet to use for uploading. Everything else seems okay; but, how would I know without a functioning csv load spreadsheet.

Is a nyone able to help?

[Paul..]

Stevo is probably your best bet...just be patient...he is on the west coast.

[manyroads]

Stevo is probably your best bet...just be patient...he is on the west coast.

Thanks, @Paul.

[Paul..]

Are you trying to do bulk uploading? Or, are you trying another path because the web-based upload is too slow?

[manyroads]

Are you trying to do bulk uploading? Or, are you trying another path because the web-based upload is too slow?

I'm trying to bulk upload (some 35 files consuming 14Gb of space) to a pre-existing folder. Doing that with their web-based upload will complete a year or two after my demise. Genealogist talk...

[Stevo]

Stevo is probably your best bet...just be patient...he is on the west coast.

Umm...probably no help from me. I've just done the point and click web interface uploads, so you've gone way past me now.

[Paul..]

It's a shame you can't use rsync like sourceforge.org

[manyroads]

Stevo is probably your best bet...just be patient...he is on the west coast.

Umm...probably no help from me. I've just done the point and click web interface uploads, so you've gone way past me now.

I guess it'll be Google time... I'll see if I can find a really big hammer and some duct tape to make it work.

[skidoo]

using pip is the python equivalent of installing .deb files from PPAs, so let's not go there, eh.
You could git clone (or download via html) that same "ia" commandline program from
https://github.com/jjjake/internetarchive
but I don't know offhand whether its requirements are available as debian packages (pkgnames may differ from the exact names of the python libraries they contain)

I found this
https://internetarchive.readthedocs.org ... t/cli.html
which redirects to this
https://internetarchive.readthedocs.io/ ... t/cli.html
placeholder page which states "this page does not exist"
so I don't have much confidence the "ia" utility is, or ever was, ready for primetime...

[manyroads]

using pip is the python equivalent of installing .deb files from PPAs, so let's not go there, eh.
You could git clone (or download via html) that same "ia" commandline program from
https://github.com/jjjake/internetarchive
but I don't know offhand whether its requirements are available as debian packages (pkgnames may differ from the exact names of the python libraries they contain)

I found this
https://internetarchive.readthedocs.org ... t/cli.html
which redirects to this
https://internetarchive.readthedocs.io/ ... t/cli.html
placeholder page which states "this page does not exist"
so I don't have much confidence the "ia" utility is, or ever was, ready for primetime...

I found a work around for my immediate challenge. I had no problem with the ia install really. I even found mention of others on debian doing somethings very similar for other python apps. Now that I've passed my urgency obstacle; I'll see what I can do.