If you exclude the deep work on models themselves and just focus on use — harnesses etc. — then the intellectual community is basically prescientific. Very little meaningful empiricism. danluu's notes on benchmarks and experimental design come to mind: https://danluu.com/exercise-7/

▲

ChickeNES an hour ago | parent [-]

This response confuses me, as I’m pretty sure OP is talking about AI use (i.e. applied use, from solving math problems to vibe coding to etc etc) not something like harness design or benchmarking? I’m also not sure what empiricism has to do with it, are non-scientists not intellectuals in your eyes?

	▲	lukasschwab 10 minutes ago \| parent [-]
		Nope. There's a lot of unscientific intellectualism out there. I don't see a lot of intellectual depth in the communities springing up around AI use — I had vibecoding in particular in mind, and Fernando's blog post directly addresses mathematics. I don't understand the contrast you're drawing. Harness design and benchmarking are major aspects of vibe coding (and solving math problems with LLMs, for that matter); I would go so far as to say harness design and benchmarking are the more intellectually serious aspects of vibecoding. Fernando is definitely aware of them as well. These just aren't meaningful substitutes for the intellectual work being diminished.