Remix.run Logo
b33j0r 2 days ago

For me, once I wanted to scale asyncio within one process (scaling horizontally on top of that), only two things made sense: rust with tokio or node.js.

Doing async in python has the same fundamental design. You have an executer, a scheduler, and event-driven wakers on futures or promises. But you’re doing it in a fundamentally hand-cuffed environment.

You don’t get benefits like static compilation, real work-stealing, a large library ecosystem, or crazy performance boosts. Except in certain places in the stack.

Using fastapi with async is a game-changer. Writing a cli to download a bunch of stuff in parallel is great.

But if you want to use async to parse faster or make a parallel-friendly GUI, you are more than likely wasting your time using python. The benefits will be bottlenecked by other language design features. Still the GIL mostly.

I guess there is no reason you can’t make tokio in python with multiprocessing or subinterpreters, but to my knowledge that hasn’t been done.

Learning tokio was way more fun, too.

ciupicri 2 days ago | parent | next [-]

GIL is not part of the language design, it's just a detail of the most common implementation - CPython.

b33j0r 2 days ago | parent [-]

Fair and accurate. But that’s pretty much what people use, right?

I am happy to hear stories of using pypy or something to radically improve an architecture. I don’t have any from personal experience.

I guess twisted and stackless, a long time ago.

miohtama 2 days ago | parent [-]

The GIL is optional in new Python versions. Downsides are legacy library compatibility and degraded single thread performance.

b33j0r a day ago | parent [-]

I mentioned subinterpreters. That’s the thing that “makes the GIL optional,” you still have to use subinterpreters.

Is this no longer true?

miohtama 30 minutes ago | parent [-]

It is no longer true. You can have free threading.

smw 2 days ago | parent | prev | next [-]

Or just golang?

iknowstuff 2 days ago | parent [-]

Segfaults

hinkley 2 days ago | parent | prev [-]

I don’t know where Java is now but their early promise and task queue implementations left me feeling flat. And people who should know better made some dumb mistakes around thread to CPU decisions that just screamed “toy solution”. They didn’t compose.