> Time to fetch version data for each one of those packages: ~12 hours (yikes)

The author could improve the batching in fetchAllPackageData by not waiting for all 50 (BATCH_SIZE) promises to resolve at once. I just published a package for proper promise batching last week: https://www.npmjs.com/package/promises-batched

▲

winrid a day ago | parent | next [-]

What's the benefit of promises like this here?

Just spin up a loop of 50 call chains. When one completes you just do the next on next tick. It's like 3 lines of code. No libraries needed. Then you're always doing 50 at a time. You can still use await.

async work() { await thing(); nextTick(work); }

for(to 50) { work(); }

then maybe a separate timer to check how many tasks are active I guess.

▲

whilenot-dev a day ago | parent [-]

Promise.all waits for all 50 promises to resolve, so if one of these promises takes 3s, while the other 49 are taking 0.5s, you're waisting 2.5s awaiting each batch.

The implementation is rather simple, but more than 3 LoC: https://github.com/whilenot-dev/promises-batched/blob/main/s...

▲

winrid a day ago | parent [-]

I know. My point is you can do better without a library.

	▲	halfmatthalfcat a day ago \| parent [-]
		Why not write all of our applications on one file? Why bother using (language specific) modules? To take your argument to the logical extreme, DRY is a fanatical doomsday computer science cult.

▲

1gn15 a day ago | parent | prev | next [-]

Worried about being rate limited or DoSing the server.

	▲	whilenot-dev a day ago \| parent [-]
		Sure, the need for backpressure occurs anyway, regardless of batching optimization. Couldn't find any specific rate limit numbers besides the one mentioned here[0] from 2019: > Up to five million requests to the registry per month are considered acceptable at this time [0]: https://blog.npmjs.org/post/187698412060/acceptible-use.html

▲

genshii a day ago | parent | prev [-]

Ah this is cool, thanks!