| ▲ | zug_zug 19 hours ago | |
Yeah the 5% rate is somewhat arbitrarily picked, but the reason I expect it not to look like go is because the machine is improving itself, so presumably it unlocks a higher "iq" which in turn will in turn mean that the types of optimizations it will be able to find will grow. Of course there is a raw hardware limit, but it's my intuition that we are under-utilizing the hardware by multiple degrees of magnitude. The whole pattern of using token prediction and realizing intelligence falls out the other end has been more or less stumbled upon by us, there's no reason to think it's optimal. | ||