Very cool! Do you have a recommended way for an agent to see an index of the books and epub links?

(I can’t quite tell if that’s an egregious abuse of the site or you’re perfectly fine to share without human eye balls hitting your www?)

Now i'm not associated with gutenberg in any form, but they do have a page for offline consumption:

https://www.gutenberg.org/ebooks/offline_catalogs.html

Perhaps you can find the information you are looking for there.

However if you plan on scraping or otherwise hitting them with a ton of traffic, consider at least to donate a good amount for the traffic you cause them. It ain't free after all.

Check out https://www.gutenberg.org/ebooks/offline_catalogs.html

Don't hit the site with agent. The section furtherst bottom machine readable.

Thanks for the answers! Found it:

> All Project Gutenberg metadata are available digitally in the XML/RDF format. This is updated daily (other than the legacy format mentioned below). Please use one of these files as input to a database or other tools you may be developing, instead of crawling or roboting the website.

And strongly consider a donation! (My addition)

https://www.gutenberg.org/ebooks/offline_catalogs.html#the-p...

not yet, but that's not a bad idea imo. Dealing with Ai crawler traffic is definitely a challenge if that's what you were referring to.

if what you want is all the text, please use the tarball or data files at https://www.gutenberg.org/cache/epub/feeds

OPDS?

[flagged]