The time it takes to get data to/from the GPU is problematic given the sample rate you need to deliver.

Anukari used the GPU but eventually abandoned it for the CPU. They have some good and informative blog posts about it.