Remix.run Logo
bgirard 5 hours ago

It's really neat that the prompt was released!

I'm curious how many unsolved problems are tried against frontier models when they come out. Are we trying every problems against every release? What is the solve success rate? Is there a sub-community within Mathematics that is coordinating this effort? How much untapped opportunity is there here?

edflsafoiewq 35 minutes ago | parent | next [-]

I find it kind of interesting the whole output wasn't released. A common criticism of mathematical writing is results are "pulled out of a hat"; you only write up a polished, final proof, but hide everything that went into developing it. It's kind of ironic the practice is even carried on when an LLM writes the proof.

emil-lp 5 hours ago | parent | prev | next [-]

The prompt was released, but not the cost of the result.

riknos314 5 hours ago | parent | next [-]

Assuming all 64 subagents were running for a full hour (the tweet states just under an hour):

  Throughput                    Output tokens   Output cost
  ----------------------------  -------------   -----------
  40 tok/s  (5.5 low)                   ~9.2M         ~$275
  55 tok/s  (5.5 base)                 ~12.7M         ~$380
  70 tok/s  (5.5 high)                 ~16.1M         ~$485
  750 tok/s (Sol Fast, $75/M)         ~172.8M       ~$13,000
Claude estimates that tool use / input tokens might add 10-15% on top of that depending on exactly how the model went about the task.

Edit: better tok/s estimate buckets based on GPT 5.5 actual speeds since I couldn't find real benchmarks on 5.6 published anywhere. Also account for Sol Fast pricing.

conradkay 3 hours ago | parent | next [-]

Sol fast isn't the Cerebras 750 tok/s version, it's just 1.5x speed at 2.5x price

I assume they didn't use the Cerebras version for this since it's probably very supply-constrained right now

throw1234567891 2 hours ago | parent [-]

But Sol is running on Cerebras. That’s the whole point of this. That’s how they get 750 tokens per second. There is no other way.

Anuiran 2 hours ago | parent [-]

Regular Sol does not run on Cerebra’s. I don’t think anyone public has access to that.

https://x.com/thsottiaux/status/2075596669958472146?s=46&t=Z...

judge2020 41 minutes ago | parent [-]

Yeah they posted an update below it

> apparently that is just Sol being Sol on fast mode, not 750 tps. :x O.o

> This is real and it is not 750 TPS. Anyone with 5.6 Sol Ultra on fast mode can reproduce this! It’s all GUI interactions with CUA. No MCP or bpy needed.

https://x.com/kimmonismus/status/2075493505011482922?s=20

3 hours ago | parent | prev [-]
[deleted]
therobots927 5 hours ago | parent | prev [-]

And not how many times it was prompted before it returned a working solution.

Or how many prior variants of this prompt were tried.

Or if proof checking software was used to hone in on the final winning prompt / LLM output.

not-a-llm 5 hours ago | parent | prev [-]

pretty sure already millions of dollars (in inference costs) were already thrown at the Riehmann hypothesis

as the models get stronger, larger amounts will be thrown at it

imagine paying "just $1 bil" to go down in history as the company who's model solved the hardest/most famous open problem in mathematics. imagine the worldwide press headlines.

as they say, the Riehmann Hypothesis is the hardest way to earn a million dollar

Frost1x 5 hours ago | parent | next [-]

I’m all for it since it’s value directly returned to humanity.

5 hours ago | parent | next [-]
[deleted]
hnisfulomrons 4 hours ago | parent | prev [-]

[dead]

CSMastermind 4 hours ago | parent | prev [-]

I mean if there's something I'd bet against being solved by LLMs in my lifetime it's that one. We truly do not have line of sight into what a proof would even look like.

blovescoffee 2 hours ago | parent [-]

Why would you bet against it being solved by LLMs? Isn't this very post proof that LLMs in an agentic harness are capable of doing real math? If you just keep cranking away at the tokens I don't see a principled argument against that leading to more solutions to unsolved math, even the hardest problems.

CSMastermind an hour ago | parent [-]

There are different classes of mathematical problems.

The one in the post definitely shows the advantages that LLMs have compared to humans for some problems but it's in an entirely different class than the Riemann Hypothesis.

Riemann is one of the most studied math problems of all times and all of humanity has basically collectively failed to make progress. The idea that there's some technique that just hasn't been tried yet (like in the post) is very very unlikely.

The general consensus is that we'll need an entirely new branch of mathematics to solve Riemann - our current tools aren't just inadequate; they're of the wrong class entirely.

I suspect inventing new branches of math will remain beyond LLMs for the remainder of my life.