Its not efficient to write low level code. Its like trying to build a web application with C.

CUDA is basically the equivalent of "hey you wanna do matrix multiply fast? just use this and we handle the rest". And its standardized, and when Nvidia comes out with new cards, all the old shit still works.