> All of these features are about breaking the coupling between a human sitting at a terminal or chat window and interacting turn-by-turn with the agent.

This means:

- less and less "man-in-the-loop"

- less and less interaction between LLMs and humans

- more and more automation

- more and more decision-making autonomy for agents

- more and more risk (i.e., LLMs' responsibility)

- less and less human responsibility

Problem:

Tasks that require continuous iteration and shared decision-making with humans have two possible options:

- either they stall until human input

- or they decide autonomously at our risk

Unfortunately, automation comes at a cost: RISK.

▲

dist-epoch 4 hours ago | parent [-]

AI driven cars have better risk profiles than humans.

Why do you think the same will not also be true for AI steerers/managers/CEO?

In a year of two, having a human in the loop, will all of their biases and inconsistencies will be considered risky and irresponsible.

▲

khafra 3 hours ago | parent | next [-]

"Did the vehicle just crash" has a short feedback loop, very amenable to RL. "Did this product strategy tank our earnings/reputation/compliance/etc" can have a much longer, harder to RL feedback loop.

But maybe not that much longer; METR task length improvement is still straight lines on log graphs.

▲

dist-epoch 3 hours ago | parent [-]

The AI has read all the business books, blogs and stories.

Unless your CEO is Steve Jobs, it's hard to imagine it being much worse than your average pointy haired boss.

	▲	rapind 2 hours ago \| parent \| next [-]
		> The AI has read all the business books, blogs and stories. This seems like a liability as most business books, blogs, and stories are either marketing BS or gloss over luck and timing. > Unless your CEO is Steve Jobs, it's hard to imagine it being much worse than your average pointy haired boss. As someone using AI agents daily, this is actually incredible really easy to imagine. It's actually hard to imagine it NOT being horrible! Maybe that'll change though... if gains don't plateau.
	▲	nprateem 2 hours ago \| parent \| prev [-]
		But they are shit. Over the last 2 days I've got bored of the predictable cycle of it first getting excited about a new idea then back peddling once I shoot it to pieces. They can't write and think critically at the same time. Then subsequent messages are tainted by their earlier nonsensical statements. Opus 3.7 BTW, not some toy open source model.

▲

jddj 3 hours ago | parent | prev | next [-]

Getting to that point is likely going to involve a lot of (the business and personal equivalent of) Teslas electing to drive through white semitrailers.

▲

3 hours ago | parent | prev | next [-]

[deleted]

▲

philipwhiuk 2 hours ago | parent | prev | next [-]

Or autonomous weapons?

▲

oblio 3 hours ago | parent | prev [-]

> AI driven cars have better risk profiles than humans.

From which company? I hope you say "Waymo", because Tesla is lying through its teeth and hiding crash statistics from regulators.