If the Google and meta engineers are not dumb how come they consistently trail behind the frontier labs and even the Chinese labs with a fraction of the funding.

Probably bad leadership

▲

hnfong 9 hours ago | parent | next [-]

I always suspect they have the most to lose if legal decisions on copyright issues don't go their way.

Imagine a scenario (theoretically possible but increasingly unlikely) where a US court decides that using "pirated" copyright data to train models is illegal. Now the AI developer has invested hundreds of billions of capital into a thing that is declared illegal and has to be scrapped.

This risk affects existing megacorps more than "startups" like OpenAI and Anthropic (and Chinese companies), because the megacorps have much more to lose. They actually have the cash to pay damages if the flood of copyright claims arrive at the door. This will not only bomb their AI development, but also the rest of their established businesses as well.

And thus I strongly suspect legal issues are holding them back a bit. Megacorps want to win the AI race, but not to the extent they stake the rest of their established business, while the newer companies' only product is AI, so they have to go all in.

Notice for example how Meta's Llama performed much more poorly after they got smacked by a bunch of lawsuits claiming that they torrented a bunch of copyright data.

(Disclaimer: I'm an outsider and everything I base my speculations on is public knowledge.)

▲

BoredomIsFun 7 hours ago | parent | prev | next [-]

they, rightfully so, have no faith in LLMs.

▲

kubb 10 hours ago | parent | prev [-]

That plus they don’t distill so they have worse RL examples.

▲

shunia_huang 8 hours ago | parent [-]

But they both spent tons of money on data collecting/labeling/generation, how is it bad compared to distillation? I thought their data are much better if they spent that much, and it seems they are stupid because with that much of resources putting in there with merely no output compared to the frontier models.

	▲	kubb 8 hours ago \| parent [-]
		Creating a RL example by hand is hundreds of times more expensive than generating one using an LLM. Of course the Chinese companies have incredibly talented researchers, and smaller, better organized org structures which account for the rest of the difference.