I don't think we get rid of the CPU. But the relationship will be inverted. Instead of the CPU calling the GPU, it might be that the GPU becomes the central controller and builds programs and calls the CPU to execute tasks.

Sounds reminiscent of the CDC 6600, a big fast compute processor with a simple peripheral processor whose barreled threads ran lots of the O/S and took care of I/O and other necessary support functions.

This will never without completely reimagining how process isolation works and rewriting any OS you'd want to run on that architecture.