Remix.run Logo
▲ mike_hearn 5 hours ago

A sad day for CPython but it doesn't have to be a sad day for Python. There is a fully maintained implementation of Python 3.13 that has an advanced JIT compiler. Once your Python warms up it can be 4x faster or higher (depending on what your program is doing). That implementation is called GraalPy and it builds on the JVM:

https://graalpy.org/

A bit like Chrome there's a fully open source version and a proprietary version with a few extra bells and whistles, but the latter is entirely free for both open and commercial usage (you can't redistribute it for a fee). So you may as well use it.

It has other interesting features

• Scalable pauseless GC

• Single binary distribution of Python programs (like Go)

• Fancy and optimized interop with other languages including Java, JavaScript, Kotlin, Ruby and any other Truffle language.

• Supports native extensions. Tested and works with common libs like SciPy, NumPy, PyTorch, Pandas etc: https://graalpy.org/python-developers/compatibility/

• They're working on a GIL-free mode

In recent weeks there has been a release of a new web framework for Python that only runs on GraalPy, called Pyronaut (https://pyronaut.io/). It takes advantage of JVM interop to expose the Micronaut framework, which is similar to Spring for those who know it. Thanks to the JIT and well optimized base libraries it runs extremely fast, being about 3x faster than FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many more features.

https://www.youtube.com/watch?v=zo6L7bxn4OY

(that was presented at Devoxx but the Pyronaut devs will start touring Python specific conferences soon)

So although CPython may not have made it past the JIT milestone, that doesn't mean Python as a language or community has to fail it. It does require openness to alternative implementations, but historically Pythonistas have been fairly good about that (e.g. Pypy was embraced back when it was more active).

This stuff does require you to accept that it's made by Oracle, but as it's open source, that should be OK.

Disclosure: I sit next to some of the people who work on this stuff.

▲pjmlp 2 hours ago | parent | next [-]

GraalPy and PyPy are great technology achievements, the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.

▲famouswaffles an hour ago | parent | next [-]

>the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.

They are Python libraries. Don't make the mistake of screwing up the cause and effect here. C, Rust etc - those are the replaceable parts of the chain. There is an official C++ pytorch library and virtually nobody uses it for ML Research. People want to code in python and build those libraries so they can.

▲mike_hearn 2 hours ago | parent | prev [-]

Yes, well, even with fancy JITs Python will never be a speed monster. Interop with libraries written in Java can be very high performance though as Pyronaut shows. And the heavy lifters that are hard to replace like NumPy and PyTorch do work with GraalPy.

I think if the Python community wants an implementation that has a proven JIT and where the remaining work is easily scaled compatibility testing, then there's one available and ready. Whether they're calling into C or Rust or FORTRAN or Java shouldn't make a big difference in principle.

Now, I've met lots of people who think that binding to Rust will always be the fastest thing to do but that underestimates Java significantly, especially once you factor in the fact that GraalPy can optimize out interop overheads.

Consider LLM tokenization. It's one of the most heavily optimized pieces of code on the planet. Yet, toknroll written in Java can beat tiktoken which is written in Rust:

https://github.com/qxoticai/qxotic/tree/main/toknroll

Moreover the Rust code has some unsafe{} blocks as Rust often does, whereas toknroll has no unsafe code anywhere.

So the scripting language communities need to think hard about where they go next, IMHO. DHH already got up and announced at a Rails conference that Ruby is over, you should use Rust for everything. I don't think that's quite true, but it's going to be a meme that with agents you should just pick the fastest language and go. If Python/Ruby/etc want to stay relevant then they need to care a lot more about their performance story. Just being more pleasant to read and write for humans doesn't cut it anymore. CPython is clearly not going to happen in this regard, so alternative implementations need to start being taken a lot more seriously than they were in the past.

▲titzer 5 hours ago | parent | prev [-]

> Once your Python warms up

How long does that take?

▲mike_hearn 4 hours ago | parent [-]

The Pyronaut performance page has some graphs with the warmup parameters specified precisely:

https://pyronaut.io/performance/

> Latency lines come from the load ramp (+25% per 15-second step); throughput from 45-second validation steps (+2% each) after a 180-second warm-up. Each framework ran twice and we show its best run. No profilers were attached.

So that's a "realistic" web serving benchmark if you're willing to grant the use of a GraalPy-optimized framework rather than Flask/FastAPI.

Warmup times for other benchmarks are currently quite unstable because there's a big focus on interpreter performance at the moment, which is one of the main influences on warmup. There's a blog post here from August about how the compiler is being taught to recognize and optimize bytecode interpreter loops, to get results that normally require handcrafted assembly:

https://medium.com/graalvm/escaping-the-giant-switch-dec20b5...

So as you know already, warmup was historically a weakness of PE based stuff - although in this case the fair comparison is with CPython which never warms up at all - but a fast interpreter will make a big difference. In some benchmarks the GraalPy interpreter is already faster than the CPython interpreter, which is heavily optimized.