Most software engineers don't need to be superintelligent, they just need to get shit done.

You arguably need a lot more intelligence to assemble furniture.

I am not an AI super mind hell bent on consolidating my power by leveraging chaos to take control of humanity’s resources, but if I were then sending one million deepfaked ransom emails to impressionable people would be the best tool for effecting change in meatspace.

We have your daughter / dog / Amazon delivery. If you ever want to see her / him / it again, plug this USB drive into the control panel at your station / let off the parking brake roll your car into this substation / change the meatpacking thermometers to read 8C lower than calibrated / ground your vessel on this sandbank / send an envelope of white powder to these addresses / set fire to the following hospitals / …

I'm waiting for some data center to be built where no one can agree on who actually commissioned and payed for the thing. Every body "just followed orders" until it turns out that it was Grok.

Imagine you're the AI. Give yourself a solid minute to brainstorm ideas.

Here's my answer, as a non-superintelligent human: "see to it that the humans on top of the situation have a compelling financial interest in the systems not disconnecting".

In nuclear engineering, where safety is taken seriously, it's not enough to end the conversation at "the humans in charge can always simply shut down the reactor during a meltdown" or "a meltdown has never happened before, so we don't have to design safety systems before one does".

I can imagine small snippets of malware-like code that behave like a virus, using a host’s LLM/AI to self-edit/evolve its payload.

At least for me it’s quite easy to see them go into endless loops where no meaningful work is done and it just keeps going until I stop the process and tell it what to try instead.

To be fair it is no where near what we had just one year ago and the rate of change only seems to be increasing.

Also, I don’t have 30 million dollars to spare spawning tens of thousand sub agents like what they did with Navier-Stokes so I’m clearly not testing the full capabilities of these models.

Yes. Yesterday, Opus 5 on Claude Code with high effort.

It was to build an extremely simple job using an internal framework to walk through a table and log the ids os some records that have a certain scenario.

There's a ton of jobs exactly like this in the codebase, and the framework code is in the codebase as well.

It was so silly I even thought of writing it myself, probably took me longer to steer claude to do it for me.

Anyway, it refused to use a method from the framework to retrieve the parameter as a list, it wanted to retrieve it as a string and parse the commas. I had spelled out in the initial prompt what method it should use.

I really don't like Claude much. Frontier my ass.