Remix.run Logo
schmidtleonard 3 hours ago

I can't believe parallel processing is still this big of a dumpster fire in python 20 years after multi-core became the rule rather than the exception.

Do they really still not have a good mechanism to toss a flag on a for loop to capture embarrassing parallelism easily?

skylurk 2 hours ago | parent | next [-]

This is one of the reasons I use polars.

ritchie46 2 hours ago | parent | prev | next [-]

Polars does that for you.

lairv 2 hours ago | parent | prev [-]

Well I think ProcessPoolExecutor/ThreadPoolExecutor from concurrent.futures were supposed to be that