> and running local ones becomes cost prohibitive.

Running local models is getting -cheaper- every month in spite of rising hardware costs due to intelligence compression innovations like Bonsai2.

I build operating systems for a living using only consumer GPUs at home.