I thought we see exp right now or indications.

Like models getting significant better every 7 month and model usage on openrouter.

https://metr.org/ << someone said though that they are biased.

Math is for sure about solving mathematical problems, mathematicians complaing about humans capability of understanding it. If you need math an LLM understands, it might not be relevant if you understand it if an LLM is getting a lot better in adjusting itself if it goes the wrong way.

And this situation is only a few weeks old. We don't know how this field continues to progress. Books like the culture series and other scifi do describe a future were humans spend their time only to understand what the machines are doing or did.

The speedup of medicines is for sure faster with. There are still old labs doing everything by hand and the neolabs doing everything a lot more automated. An AI can do experiments 24/7 and can scale with the hardwarelabs, but it also can hold the whole research field in context and it might be a lot faster of choosing the better experiments or the right ones to begin with.

In software developmet (my field) i can do today alone what you needed a Team for. Yeah it doesn't help the field of software engineering if you need 5 times less people and you don't have any motivation AT ALL to teach someone avg. We are already planning to remove one member of our team due to AI. And we will not extend the contract of 2 consultants.

It just doesn't make sense anymore. If i explain in a ticket a task well enough for someone who has not been in our team for years, the AI is already better in it. We don't struggle with AI output, we struggle more with not having a Monorepo.

Give an LLM access to a GPU system or a GPU cluster. It immediadly feels complelty different then running it on some laptop. It can now do simulations, a lot more and faster experiments etc. I had this epiphany just a few month ago. I always used my laptop for projects (it needs very little ram, i can quickly check what it is doing and i actually do use it primarily when im on holidays) until I thought "let me use claude to do some gaussian splatting experiment its always so cumbersome to use these tools" and it changed my perception immediadly.

Also, the frontier labs are probably always internally using a model more advanced than what has been publicly released.

I have to imagine if I worked for one of these companies on the frontier models it would be very difficult to leave for that reason. Thee pay is also pretty good.

Nice description

Welcome to Singularity.

Now imagine feeling like this, except not just about "AI events" but about everything around you.

Yes, and not very helpful without knowing how many agents were used by the custom-built harness. The 30 hrs vs. 11 days means nothing without at least some token-to-token comparison. Without that, they are pumping up their new model with half data - better to just say "we solved it in 30 hours" and who cares about your internal, custom model.

All the human engineers will be in harnesses in the future.

Within three years most software developer jobs will consist of taking instruction from the AI systems. The humans won't see it that way, but that's what it will actually be. Realistically the thing that will happen is the humans will be put into harnesses (cubicles) and prompted by the AI (I need you to plug that server in; I need you to give me access to such and such; now do this, then do that). Most tech jobs will consist of AI telling the human what the AI needs the human to do next, de facto the human will be in a harness and will be getting told what to do.

> We've sort of entered this twilight zone where most humans don't have the time, expertise, or moral compass to be able to use these models.

Sounds a bit like self aggrandizing elitism to me. I can imagine some self aggrandizing nerds saying the same thing 50 years ago:

> We've sort of entered this twilight zone where most humans don't have the time, expertise, or moral compass to be able to use these IBM PCs.

Now everyone use computer.

We're all harness engineers already

> We're all going to be harness engineers in the future

No, that will be the model's job. We will largely be unemployed. Of course, we can surely depend on the largesse of Altman and Amodei to keep us fed and sheltered.

Maybe that's true, and maybe you're an outlier. But from what we've been seeing, the frontier LLMs have torn apart many codebases, finding bugs littered everywhere, some over a decade old.

At this point it seems impossible for human to write better/more efficient code than a swarm of agents -- to be clear, multiple coordinated LLMs, not one-shot builds with a single agent.

>Gotta be honest, LLMs are still subpar compared to what I do.

Im sorry... what?

You are replying to this:

> Our first formalization of Fermat’s Last Theorem, performed by an unreleased research version of Claude, took 11 days. It relied on a purpose-built harness along with guidance from a human researcher. By contrast, a team of 1,000 Fable 5.2 agents fully formalized the theorem in just 30 hours, with minimal human intervention. Fable 5.2 started from minimal prompts, built its own ways to coordinate, and used no mathematical references, only a list of intermediate theorem statements to score progress. It needed one-third of the action tokens as our first formalization and produced a proof seven times shorter. As can be seen in the figure below, Opus 5.5 (with the same 1,000-agent setup) completed only 26% of the formalization in 18 hours, and barely made any more progress after 30.

Maybe this is how it's gonna be.

AI discovers 500 things per day.

People mine them for interesting results.

Devote precious time (in the face of tons of discoveries) to validate the most compelling ones.

If agents actually can continue to produce tons of results with real world implications, and they arent generally found to be shit, I could see that becoming the job of experts. AI discovery validators.

I'm using Opus 5.5 and I cant help but think that I might very well be one of the few people who do so in my local area ( developing country, one of the poorest province ).

Had Opus make software for some local business and seeing it blew people's mind.

20$ aint cheap for us but I'm actually considering upgrading to the 100$ tier.

Am just jelly people are using 100-200$ subscription ^_^

I’d be happy even without having access to a billion dollar datacenter, if I could use local models properly, but even that requires some serious hardware that is out of my reach. We live in weird times: I have lots of hope for the future, but meanwhile AI access is pretty iniquitous indeed.

apply at one of the frontier labs

Dune books by Frank Herbert never actually went into what "Butlerian Jihad" was.

The semi-canonical Dune Encyclopedia (disregarded by Frank's son) differs vastly from prequels and generally points to it being a self-inflicted wound on humanity that opened the doors to widespread return of feudalism and essentially stopped humanity for a 10,000 years

It's how RLVR works, when you have a task with a verified answer, you rerun the model on it and promote the reasoning chains that reach the answer. That way you can hill climb model capabilities on any problem you find, which generalises to new problems to solve, which you can then hill climb again and so on.

So since they are doing that task anyway (re-solving the problem increasingly more efficiently), this is a easy check point to publish to show how far they've gotten in that process.

... as a test / benchmark? Hard to make a comparison without a control...

This was likely already the case in the ancient world. It was certainly the case by the mid to late 15th century.

The older, original definitions of "singularity" were less about "herp derp superintelligent AI blah blah".

and just about "we cannot really predict beyond certain things" (Vinge's "Singularity" (1993) with its "opaque wall across future")

Amusing, but this is still a qualitative change from just a few months ago.

It had seemed like progress was slowing last year as the juice ran out on moar parameters. We're well past that.