Remix.run Logo
vintermann 2 hours ago

Probably because it's expensive.

But I wish there were more "let's scale this thing to the skies" experiments from those who actually can afford to scale things to the skies.

yorwba 35 minutes ago | parent [-]

Scaling laws mean that there's not much need to actually scale things to the skies. Instead, you can run a bunch of experiments at small scale, fit the scaling law parameters, then extrapolate. If the predicted outcome is disappointing (e.g. it's unlikely to beat the previous scaled-to-the-sky model), you can save the really expensive experiment for a more promising approach.

It would certainly be nice though if this kind of negative result was published more often instead of leaving people to guess why a seemingly useful innovation wasn't adopted in the end.