This is probably part of why machines are doing so well at counterexamples. They have no aesthetic commitment to the conjecture and no embarrassment about producing something ugly

▲

tantalor 4 hours ago | parent | next [-]

That's not why.

It's because counterexamples are easy compared to proofs which require new mathematics.

GenAI is great at combining existing things in new ways (interpolation). It's terrible at creating new things from scratch (extrapolation).

▲

jebarker an hour ago | parent | next [-]

This is the wrong way to think about mathematical (or any other kind of) creativity in my opinion. In the extremely high-dimensional space of "ideas" (whatever that means) there are almost certainly profound ideas that are the interpolation of existing knowledge, i.e. the curse of dimensionality. It's not at all clear that you need to extrapolate from existing knowledge to be creative.

▲

vonneumannstan an hour ago | parent | prev [-]

>GenAI is great at combining existing things in new ways (interpolation). It's terrible at creating new things from scratch (extrapolation).

I think you believe a fallacy about how human cognition works if you think we actually do something different than interpolation

▲

datsci_est_2015 an hour ago | parent [-]

I think this is somehow related to GenAI’s issue of “confident incorrectness”. I’ve been trying to prompt GenAI with some highly challenging prompts recently, especially terse ones:

> Which Anjunabeats and Anjunadeep compilations have the most breakbeat tracks?

GenAI was able to give me breakbeat tracks that were on the Anjuna labels, but it completely flubbed on whether and which those tracks were on compilations. It was very confidently, mostly incorrect.

I think this is somehow isomorphic to the “interpolation vs extrapolation” issue. GenAI is forced to generate an answer, and there’s no mechanism by which we can interrogate the model for its confidence (at least, that’s my understanding).

And yet, I asked “How confident are you in your answer along different dimensions, ie whether those tracks are breakbeat, whether they are Anjuna tracks, or whether they are on those compilations?”

And its answer was surprisingly satisfying, it only gave 30% confidence that the tracks were on the compilations, which is where it flubbed.

Anyway, personal observations, GenAI is meant to be interacted with, not just a single prompt black box.

	▲	gwerbin 5 minutes ago \| parent [-]
		Out of curiosity which model & interface did you use? I'm starting to think that, for IR tasks like this, the number one differentiator among models and harnesses is the ability for the model to look at its available evidence and conclude that it doesn't know the answer. It depends a lot on how the chain of thought goes. I have seen a lot of newer models try to do this in their thinking traces, and it seems very hit or miss. Even Opus 4.8, GPT 5.5, and Gemini 3.1 Pro (haven't tried 5.6 yet) confidently make up BS sometimes and needed to be reminded to verify thnings instead of inferring or relying on "memory" from training data. IMO harnesses need some kind of built in "are you sure about that?" loop checkpoint that pauses the main interaction and asks the LLM to evaluate the strength of evidence for a claim it's about to make. It seems almost unreasonable to expect an LLM to next-token-predict its way to such a conclusion, reliably, without prodding. That or we just need another year of RLHF and another 250 billion parameters, IDK.

▲

refactor_master 5 hours ago | parent | prev | next [-]

This reminds me of the Go Grandmaster speaking out after losing to AlphaGo, that the model has no sense of "aesthetic play", as long as it would lead to a win within the rules.

▲

randusername 3 hours ago | parent | next [-]

I thought there was a lot of buzz about AI creativity after the infamous move 37 in that series?

▲

voxic11 3 hours ago | parent [-]

Yes but it was so shocking because it was such an inhuman, "unaesthetic" play. It was considered to be "creative" in that no human would have thought of making the move, so it can't simply be copying human play.

	▲	jebarker an hour ago \| parent [-]
		Indeed. To get "aesthetic" play you would have to mimic real human play for an interesting reason: the space of possible Go configurations and games is so mind-bogglingly vast that all of human history has only ever seen a infinitesimal sample. So any sense of aesthetics is just a consequence of chance and memetics. AlphaZero is effectively exploring a new branch of Go history so its aesthetics are completely different. Although maybe that just shows that the idea of aesthetics here isn't very meaningful.

▲

nh23423fefe 3 hours ago | parent | prev [-]

if humans could do it, it would be called beautiful. defining something to be ugly is cope

▲

OscarCunningham 9 hours ago | parent | prev [-]

They're trained on human data. I would expect them to emulate human biases as closely as possible.

▲

avianlyric 6 hours ago | parent | next [-]

This is where harness, and the fact that a machine can be endlessly prompted to try again comes in.

Even if an LLM starts by pursuing things that follow human bias, continuous failures and re-prompting to try something different will eventually force it to consider things outside of what ever biases it has.

You can do the same thing to a human. But most people would consider it unethical to lock someone in a box and force them to keep trying to solve the same problem over and over again until they figure it out.

▲

2b3a51 6 hours ago | parent | prev | next [-]

Your comment stopped me in my tracks a little bit.

Is a 'bias' in a piece of writing generally a property of word to word choice and sentence to sentence construction or is it something more nebulous? Especially in terms of the appreciation of mathematics and someone's hesitance about publishing a mathematical argument they think is ugly or brute forced in some way.

	▲	sebastiennight 4 hours ago \| parent [-]
		> Is a 'bias' in a piece of writing generally a property of word to word choice and sentence to sentence construction or is it something more nebulous? You might be fascinated when you read the story of Golden Gate Claude: https://www.anthropic.com/news/golden-gate-claude

▲

catigula an hour ago | parent | prev | next [-]

No, they are not. Specifically, this was a method based specifically on learning from scratch, like most modern AI models.

Why do you think it's called Alpha ZERO?

▲

voidUpdate an hour ago | parent [-]

Programmers love to give things names that they think sound cool

	▲	catigula an hour ago \| parent [-]
		That's nice. It's called Alpha Zero specifically because it was trained from scratch - zero, not on human data.

▲

SiempreViernes 8 hours ago | parent | prev | next [-]

Is it? I'd expect most of the training set to be synthetic data extrapolated from a small set of human authored texts.

	▲	TeMPOraL 5 hours ago \| parent [-]
		Most of the training set is half of the Internet. LLMs are pre-trained on general set of human biases and patterns of thinking.

▲

epolanski 4 hours ago | parent | prev [-]

It's more complex than that, especially as post training is often goal based.

	▲	OscarCunningham 4 hours ago \| parent [-]
		I wouldn't have expected that there was post training specifically on the issue of looking for proofs vs counter examples. But it might be that other post training has a side effect of making AIs better at looking for counter examples. I wonder if these agents are overall less biased and more rational than humans. Can you expand on what you mean by goal based training?