| ▲ | gmaster1440 4 hours ago | |||||||
I think we're basically agreeing here. Your point (if I'm reading it right) is that taste and discernment do scale, but the gains come through pretraining/parameter scaling, which is slow and expensive compared to the fast, cheap wins in math/coding from smaller models. So taste is more of a lagging indicator of scale. it improves, but it's the last thing people notice because the benchmarkable stuff races ahead. Which also means taste isn't really a moat, just late to get commoditized. | ||||||||
| ▲ | gwern 3 hours ago | parent [-] | |||||||
My point is more that since you can expect taste's commoditization to lag behind for deep fundamental reasons, then taste does serve as a moat. Just perhaps a weaker one than one would naively expect, and where you will have to frantically keep investing in it to stay ahead of the LLMs slowly catching up, as opposed to a permanent lock-in you can lazily monopolistically coast on indefinitely. (I'm reminded of Neal Stephenson's La Brea tarpit analogy for open source vs proprietary software in _In The Beginning was the Commandline_.) | ||||||||
| ||||||||