I am more worried that an LLM company could destroy your entire company by removing access or losing enough cash to go under.

AWS, Azure, GCP, etc could already do this more effectively. Your bank even more effectively.

At least for LLM absolute worst case scenario you can run models yourself.