| ▲ | swatcoder a day ago | |||||||
Do you think there might be an approval process to navigate when experiments costs might run seven or eight digits and months of reserved resources? While they do have lots of money and many people, they don't have infinite money and specifically only have so much hot infrastructure to spread around. You'd expect they have to gradually build up the case that a large scale experiment is likely enough to yield a big enough advantage over what's already claiming those resources. | ||||||||
| ▲ | dpe82 17 hours ago | parent | next [-] | |||||||
I would imagine they do not want their researchers unnecessarily wasting time fighting for resources - within reason. And at Google, "within reason" can be pretty big. | ||||||||
| ||||||||
| ▲ | nl 14 hours ago | parent | prev [-] | |||||||
I mean you'd think so, but... > In fact, the UL2 20B model (at Google) was trained by leaving the job running accidentally for a month. https://www.yitay.net/blog/training-great-llms-entirely-from... | ||||||||