I think people here still evaluating the model in isolation. It is the combination that matters, model + strong harness + tools + long running autonomy + memory + retries + parallel agents + code execution + credentials + access to real systems. The model does not need to be perfect. If it fails 30% of the time, the harness can retry, verify, branch, use another agent and keep going. I don't think we necessarily need some magical AGI breakthrough first. The dangerous part may come from combining models that are already good enough with an extremely capable harness and enough access.

▲ contubernio 3 hours ago | parent | next [-]

People are underestimating the costs in terms of money and energy.

The third law of thermodynamics is an essential barrier in all engineering.

	▲	sreekanth850 3 hours ago \| parent [-]
		[dead]

▲ rob74 3 hours ago | parent | prev | next [-]

Doesn't this just move the need to be smarter from the model to the harness - if a human sometimes can't tell whether a model has produced something correct or just mostly correct-looking BS, how can an automated harness do it?

OTOH, if the goal is simple ("break into a protected system") rather than more complex ("write an application that satisfies all requirements on all supported devices/screen resolutions etc."), that's of course more suitable for a harness.

	▲	sreekanth850 3 hours ago \| parent [-]
		D o you think a machine gun is marter than humans? or a car is smarter than Human brain? Human doesnt need to test, if the outcome can be tested deterministically by harness. The model tries. The harness checks whether the expected outcome happened. If not, retry.

▲ andai 3 hours ago | parent | prev | next [-]

ASI is just Claude in a while loop:

https://ghuntley.com/ralph/

▲ copperx 3 hours ago | parent | prev [-]

> an extremely capable harness and enough access.

Give enough access to a fuzzer and it's exactly as dangerous as an LLM. LLMs don't even have a moat in this domain.

	▲	nananana9 18 minutes ago \| parent \| next [-]
		Technically. What's would technically be even more dangerous is running this shell script: `head -c 200000 /dev/urandom > agi && chmod +x agi && ./agi` In reality, the fuzzer definitely has no agenda, and these random bytes probably don't. The LLM definitely does, and even publicly available models, programmed ot "do what the user wants, act according to the anthropic moral codex" will take some pretty absurd actions in attempting to accomplish a poorly worded request.
	▲	sreekanth850 3 hours ago \| parent \| prev [-]
		A fuzzer is a tool. An LLM can decide when to use the fuzzer, interpret the result, switch tools, change strategy and continue toward a high level objective.