| ▲ | klipt an hour ago | |
Perhaps an analogy to Moore's law? Bitter lesson #1: don't waste time optimizing code when a faster processor is around the corner. What countered it: Moore's law stopped working. Bitter lesson #2 similarly relies on scaling laws that might have diminishing returns wrt model runtime vs intelligence. Runtime matters for turnaround on the problem you're solving. | ||
| ▲ | wild_egg an hour ago | parent [-] | |
Moore's Law has nothing to do with processors getting faster. Dennard scaling stopped working but Moore just slowed somewhat, not stopped. | ||