In addition to that, while it is possible to hit a delay and run out of buffer because memory access is slow (the most obvious would be if the input got swapped to disk at an inopportune moment), but the audible effect is really obvious. This isn't some subtle "oh my music sounds ineffably worse" effect, it's "my computer is glitching and my music is unlistenable."