> LLM's - to date - seem to require massive capital expenditures to have the highest quality ones

There are near-SOTA LLM's available under permissive licenses. Even running them doesn't require prohibitive expenses on hardware unless you insist on realtime use.