Lots of progress recently in formally verifiable domains. Really curious if this is a product of the machine learning folks behind most large language models being much more likely to have a background in math, physics, etc. Or if it is that large language models have some fundamental limitations in the more "scruffy" sciences like biology.

My guess is that in formally verifiable domains you can use RL-esque processes to get lots of good high-quality training data with reward signals (does it verify?), but the same does not apply to "scruffier" work. I would also imagine that the data volume of high-quality scientific work in "scruffy" domains is too low for the large language models to have a good manifold for "high-quality biology research". Would love to be proven wrong if anyone knows of anyone doing great scientific work with large language models in non-verifiable domains though.

▲

serioussecurity an hour ago | parent | next [-]

It's not just RL.

Many of the harnesses used to search for solutions rely on quick/cheap correctness checks to discard incorrect hypotheses. With something like biology, you might need to run a wet lab experiment to reject an idea. This can add days per iteration, slowing progress down by a factor of 100 or more.

The sad reality is that many computational approximations for biological phenomenon are relatively weak and lack the predictive power we need.

	▲	szvsw 21 minutes ago \| parent \| next [-]
		Is there a path where compute gets poured into developing, and then eventually using, computational models that supersede the shortcoming of the existing computational approximations you mention, which thus makes computational analysis and developments possible at rapid scale? In an area I am more familiar with though still not my domain: when the stiffness matrix and computational linear algebra methods collide, it unlocks a world of computational structural design and analysis by commoditizing the underlying mathematics. This is a simple example, but you get the idea. Obviously with CFD, mechanics, etc there is always massive work being done to try to unlock new computational methods, faster/parallelizable computational methods, differentiable ones, etc etc, but the general feeling is that the core physics are well modeled (NS millennium problem not withstanding) and are not lacking in predictive power for useful applications. I would have assumed that the same would be generally true for computational biology etc, so just curious to hear you elaborate more. It seems plausible to me that turning the brute force of “the models” (as in, swarms of AI compute) to work on/develop/solve/address whatever representational problems there are in the computational approximations/numerical methods you mention, particularly if that is a bottleneck to performing more extensive methodological/experimental exploration in silico to enable better exploitation/use of limited resources in wet lab. Again, I know nothing about computational biology, just stating my intuition and curious what your take is. Probably the limits of my knowledge can be demonstrated by the fact that the only modeling methods that come to mind as being vaguely relevant would be, like,… molecular dynamics, maybe discrete element method in a weird way but doubtful, and various pop science understandings of protein folding etc.
	▲	Frost1x 36 minutes ago \| parent \| prev [-]
		I suspect the line forward is generating improved predictive models and implementations of those. It’s something LLMs can aide with, although they’ll have slower progress than domains that do already have robust, cheap validation checks. That’s probably why those domains are being targeted (they are the paths of least resistance to successes, which is why they’re being pursued). With that said part of the reason some domains may not have robust cheap verifiers is because they’re difficult to produce already, so it may be a bigger lift.

▲

Creamsicle47 an hour ago | parent | prev | next [-]

It's because LLMs provide an efficient way of optimizing a complex, non-linear search space when there's a way to measure progress. The less verifiable something becomes, the harder it is to optimize (but not impossible).

▲

rcpt an hour ago | parent | prev | next [-]

It's the limitations for sure.

AI has dominion over the digital world, when it can verify things in Lean it's able to crush problems like they're a game of Go.

But when the real world and digital world don't quite connect AI is bottlenecked.

	▲	crabmusket 38 minutes ago \| parent [-]
		Does this imply a forthcoming wet lab boom comparable to the data centre boom?

▲

itchyjunk an hour ago | parent | prev | next [-]

https://www.anthropic.com/research/claude-scorpion-toxin-bin...

▲

kypro 33 minutes ago | parent | prev [-]

Knowledge is surprisingly transferable.

Obviously it's easy to understand why having strong abilities in maths and coding can help with solving problems in physics. But it's also true of knowledge and mental models more broadly.

For example, say somehow you managed to train a frontier model without any data about elephants. If you did this it would be relatively easy to later train elephants into the weights. You'd basically just need to train it about elephants at a very superficial level. After all, it it already knows what a mammal is. It knows what legs are. It knows what eyes are. It understands how it's digestive system works. All the things that makes the biology of elephants difficult doesn't need to be learnt from scratch.

I assure you people on HN will be surprised at how quickly frontier models will become competent in domains that are harder to RL. They won't need anywhere near as much data as people currently believe.

	▲	Kotlopou 28 minutes ago \| parent [-]
		Has anyone done an experiment like this? Do you have any papers to link to? This sounds quite important!