Remix.run Logo
hunterpayne 2 hours ago

Then you don't understand Machine Learning in any real way. Literally the 3rd or 4th thing you learn about ML is that for any given problem, there is an ideal model size. Just making the model bigger doesn't work because of something called the curse of dimensionality. This is something we have discovered about every single problem and type of learning algorithm used in ML. For LLMs, we probably moved past the ideal model size about 18 months ago. From the POV of something who actually learned ML in school (from the person who coined the term), I see no real reason to think that AGI will happen based upon the current techniques. Maybe someday. Probably not anytime soon.

PS The first thing you learn about ML is to compare your models to random to make sure the model didn't degenerate during training.

Lerc an hour ago | parent [-]

Um, what? Are you interpreting scaling to mean adding parameters and nothing else?

I'm not entirely sure where you get your confidence that we've past the ideal model size from, but at least that's a clear prediction so you should be able to tell if and when you are proven wrong.

Just for the record, do you care to put an actual number on something we won't go past?

[edit] Vibe check on user comes out as

    Contrarian 45%
    Pedantic 35%
    Skeptical 15%
    Direct  5%
That's got to be some sort of record.