You would need fine-grained hardware configuration from the software based on that very software semantics and task. If ever possible in a shared hardware environment.

Video game consoles with shared GPU(for 3D) and CPU had to chose: favor the GPU with high bandwidth and high latency, or the CPU with low lantency with lower bandwidth. Since a video game console is mostly GPU, they went for the GDDR, namely high bandwidth with high latency.

On linux, you have the alsa-lib which does handle sharing the audio device among the various applications. They had to choose a reasonable default hardware configuration for all: it is currently stereo 48kHz, and it is moving to the 'maximum number of channels' at a maximum of 48kHz with left and right channels.