I feel like this could unlock some huge performance gains with Python. If you want to truly "defeat the GIL" you must use processes instead of threads. This could be a middle ground.

▲

hun3 4 hours ago | parent | next [-]

This is exactly what subinterpreters are for! Basically isolated copies of Python in the same process.

https://docs.python.org/3/library/concurrent.interpreters.ht...

If you want a higher-level interface, there is InterpreterPoolExecutor:

https://docs.python.org/3/library/concurrent.futures.html#co...

▲

short_sells_poo 4 hours ago | parent | prev [-]

How would this really help python though? This doesn't solve the difficult problem, which is that python objects don't support parallel access by multiple threads/processes, no? Concurrent threads, yes, but only one thread can be operating on a python object at a time (I'm simplifying here for brevity).

There are already means of passing around bulk data with zero copy characteristics in python, but there's a lot of bureaucracy around it. A true solution must work with the GIL (or remove it altogether), no?

	▲	jer-irl 4 hours ago \| parent [-]
		I'm not familiar with CPython GC internals, but I there there are mechanisms for Python objects to be safely handed to C,C++ libraries and used there in parallel? Perhaps one could implement a handoff mechanism that uses those same mechanisms? Interesting idea!