You really don't need to be large. $100k can buy you a lot of compute and it's less than hiring an engineer. With that kind of money you can build an LLM server for a dozen people.

One engineer's salary to accelerate a team of twelve is so cheap you can't afford not to.

Open models on-prem is the future, not a single doubt in my mind.