You basically need both. Parameters and good post training. If you keep on growing both, you’ll have good models.

LLMs are still surprisingly “easy”. You need maybe a couple dozens of right people, a lot of good quality data and a lot of GPU that you know how to operate. There’s relatively little “secret sauce” needed.

▲

FergusArgyll 2 hours ago | parent [-]

I think there's still a ton of secret sauce needed for serving them economically

	▲	justapassenger 2 hours ago \| parent [-]
		Sure, same for building a model in an economically sustainable way. But barier to entry is surprisingly low (expect for the huge amount of cash, of course). That’s fairly surprising, given how extremely powerful that tech is. 10 years ago it was super hard to have usable “frontier” ML. You needed very complex data warehouse, feature engineers, feature stores, multi level ranking, calibrations, tons of different model architectures, etc, etc. Each by itself was extremely hard engineering problem and really only handful of companies could deal with that complexity. With LLMs, 95% of that is gone, infra to support them is greatly simplified. Of course, to make really reliable, performant, user friendly, etc - you still need to a lot of engineering. But it’s very different challenge.