Depends upon the intelligence vs compute scaling law— which I think no one really knows. Pretty likely to be some degree of diminishing returns, but how much? Is it logarithmic, inverse quadratic, …

If training models gets way cheaper, I would expect the diminishing returns to get steeper too.

▲

pixl97 4 hours ago | parent | next [-]

And you're right, no one has any clue what the limits of intelligence are. Though to me it seems odd that humanity has reached the pinnacle of it in the last million years or so after a few billion years of lifes development. Just seems improbable we are close to the limits.

▲

trhway 12 hours ago | parent | prev [-]

>Pretty likely to be some degree of diminishing returns

intelligence may be different. If we look at biological brains - do we get diminishing returns or completely opposite scaling law when we compare our brain against say gorilla's ?

▲

Vetch 8 hours ago | parent [-]

Interesting thought to consider in principle but fails because gorilla brains continued to evolve too, just along a different path. They're not snapshots of ancestral species locked in time.

	▲	hedora 3 hours ago \| parent [-]
		Also, it’s definitely diminishing returns, by weight, at least. Architecture / biological structure matters more. I’d expect weight and wattage to be proportional for animals, at least.