Remix.run Logo
pona-a 4 hours ago

It used to be OpenAI was a real research organization that wrote real open-access papers that aren't marketing brochures, and when they did large training runs, they released all artifacts including model weights. Now certainly they are anything but. We haven't learned learned anything meaningful about ML from OpenAI since GPT-3 was released.

Their open-weights competitors like Facebook can at least claim some kind of public benefit, but it's still just running a well-understood algorithm on dubiously obtained data with longer and longer runs, give or take some inconsequential architectural tweaks.

Anthropic's mechanistic interpretability work is the most "lab-like" of these, but it's still just secondary to selling subscriptions and fear-mongering for regulatory capture/investment/publicity.