Isn't it limited because TSMC is just so good at their job? I thought I read places that their error rates are lowest in the industry.

Which leads to multiple questions - if NVidia can't manufacture elsewhere, will the best in the industry get much more expensive? is the GPU value in CUDA or TSMC?

At best, the two companies are locked into a monopsomy/monopoly market for the next 2 years where TSMC needs NVidia to sell chips and NVidia needs TSMC to make them - in the long term, these arrangements are rarely stable. NVidia wants more manufacturing partners, and TSMC wants more GPU makers.