You're right about this, but I don't regret my choice of words.

I admit that I don't think formal "papers" deserve their elevated status in today's information economy.

I think they add a whole lot of ceremony that usually results in a format that's harder to understand.

So me calling this a "paper" was a little bit of a dig!

Of course, I agree with you in many respects - but I’ve found the academic paper lifecycle to be tremendously helpful in terms of polishing my contributions (admittedly, not in ML - but in PL / systems) and avoiding content which could be construed as “misleading”

You, being of sound mind, tried the technique and drew reasonable conclusions … but just look at the original post for this blog post: there are a large number of responses by people who clearly don’t understand the technique, missed the experimental weaknesses, couldn’t place the post into the lineage of mech interp results that preceded it … and will leave the discussion with the wrong impression.

In your mind, whose fault is that? Is that that the unavoidable cost of a field, moving too rapidly to engage in peer review?

In its ideal form, the paper lifecycle is not a gate, but a sharpening stone.

▲

simonw 2 hours ago | parent [-]

I think we have a messy version of peer review playing out right now.

I'm not convinced the alternative - delaying publication by 6 months, keeping the new information available only to a much smaller group until then - would have been better.

All of that said, I did just update my own post to include "The paper (OK, the authors classify is as a “blog post”, but it’s paper-shaped) found that frontier models..." inspired by this conversation.

	▲	mccoyb 2 hours ago \| parent [-]
		Agreed! Now if the authors update their post in response — we’ll have a proper venue on our hands!