For high performance work, gpus have replaced cpus a long time ago.

Not for all definitions of HPC, though.

No one's doing database management on GPUs. No one's scraping data on GPUs. Can't run VMs on GPUs. Can't run web servers on GPU...