TFA isn’t about consumer usage, it is about training the next generation models. An interesting thought I heard recently is that a SOTA model has about the same parameters as the number of synapses in a Golden Retriever’s brain that are not dedicated to biological processes like breathing. Most of that should be wrapped in double quotes, don’t take it literally! But that number is about 100x lower than the same synapses in a human brain.

If the next order of magnitude costs 40B, I wonder if it’s even possible to get to the one after.