I'm very interesting in this field (realtime audio + GPU programming). How do you deal with the latency? Do you send or multiple single vectors/buffers to GPU?

Also I think because samples in one channel need to be processed sequentially, does that mean mono audio processing won't benefit a lot from GPU programming. Or maybe you are dealing with spectral signal processing?

Yes, I process per-buffer, same as on CPU.

You need to find parallelism somewhere to make it worth it. This can be multiple independent channels/voices, one large simulation, one high quality simulation, a large neural network, solving PDEs, voxel simulation (https://www.youtube.com/watch?v=1bS7sHyfi58), additive synthesis, a multitude of FFTs...