Not all AI uses LLM, and for some common LLM applications like summarization and translation you can already use CPU only models. The government, or even your average employer, is not going to need a lot of AI video generation or other really GPU intensive tasks. Prompt processing is currently more GPU oriented, but I don't see it as an impossible challenge given, say, 10-15 years.

Also, CPU-only doesn't necessarily mean "on your own computer". You can easily have 100 TB RAM in a couple of racks.