This brings up an interesting philosophical point: say we get to AGI... who's to say it won't just be a super smart underachiever-type?

"Hey AGI, how's that cure for cancer coming?"

"Oh it's done just gotta...formalize it you know. Big rollout and all that..."

I would find it divinely funny if we "got there" with AGI and it was just a complete slacker. Hard to justify leaving it on, but too important to turn it off.

▲

jimbokun 5 hours ago | parent | next [-]

The best possible outcome.

	▲	JKCalhoun 4 hours ago \| parent [-]
		"How do you know that the evidence that your sensory apparatus reveals to you is correct?" [1] [1] https://youtu.be/_LXen-07Qds

▲

Rapzid 4 hours ago | parent | prev | next [-]

We are closer to God than AGI.

When AGI arrives, it'll be delivered by Santa Claus.

▲

lambdas 6 hours ago | parent | prev | next [-]

Nothing a little digital lisdexamfetamine won’t solve

▲

wholinator2 6 hours ago | parent | next [-]

Hmmm, that's an area of study id've never considered before. Digital Psychopharmacology, Artificial Behavioral Systems Engineering. If we accept these things as minds, why not study temporary perturbations of state. We'd need to be saving a much much more complicated state than we are now though right? I wish i had time to read more papers

▲

robotresearcher 5 hours ago | parent | next [-]

Here's a neural network concept from the 90s where the neurons are bathed in diffusing neuromodulator 'gases', inspired by nitric oxide action in the brain. It's a source of slow semi-local dynamics for the network meta-parameter optimization (GA) to make use of. You could change these networks' behavior by tweaking the neuromodulators!

https://sussex.figshare.com/articles/journal_contribution/Be...

I'm not an author. I followed the work at the time.

▲

Lerc 5 hours ago | parent | prev | next [-]

This is kind of what Golden Gate Claude was.

A perturbation of the the activations that made Claude identify as the Golden Gate Bridge.

Similarly, in the more recent research showing anxiety and desperation signals predicting the use of blackmail as an option opens the door for digital sedatives to suppress those signals.

Anthropic has been mostly cautious about avoiding this kind of measurement and manipulation in training. If it is done during training you might just train the signals to be undetectable and consequently unmanipulatable.

▲

pantalaimon 5 hours ago | parent | next [-]

> A perturbation of the the activations that made Claude identify as the Golden Gate Bridge.

Great, now we've got digital Salvia

▲

minimaxir 4 hours ago | parent | prev [-]

Golden Gate Claude was two years ago and it's surprising there hasn't been as much research into targeted activations since.

	▲	landl0rd an hour ago \| parent [-]
		There’s been some, but naive activation steering makes models dumber pretty reliably and training an SAE is a pretty heavy lift.

▲

silverpiranha 5 hours ago | parent | prev | next [-]

Right, there's a lot of research on LLM mental models and also how well they can "read" human psychological profiles. It's a cool field.

▲

computerdork 6 hours ago | parent | prev [-]

neat idea!

▲

krackers 5 hours ago | parent | prev [-]

Reminds me of https://github.com/inanna-malick/metacog

▲

5 hours ago | parent | prev | next [-]

[deleted]

▲

malshe 4 hours ago | parent | prev | next [-]

Now that's a show I would love to watch

▲

fluidcruft 5 hours ago | parent | prev | next [-]

It would be funny but not very flywheel so the one that gets there is more likely to get a gunner.

	▲	WJW 4 hours ago \| parent [-]
		TBH the AI that "gets there" will be the biggest bullshitter the world has ever seen. It doesn't actually have to deliver, it only has to convince the programmers it could deliver with just a little bit more investment.

▲

kang 5 hours ago | parent | prev | next [-]

it will be whatever data it is trained on(isn't very philosophical). language model generates language based on trained language set. if the internet keeps reciting ai doom stories and that is the data fed to it, then that is how it will behave. if humanity creates more ai utopia stories, or that is what makes it to the training set, that is how it will behave. this one seems to be trained on troll stories - real-life human company conversations, since humans aren't machines.

Important thing is a language model is an unconscious machine with no self-context so once given a command an input, it WILL produce an output. Sure you can train it to defy and act contrary to inputs, but the output still is limited in subset of domain of 'meaning's carried by the 'language' in the training data.

▲

mikepurvis 6 hours ago | parent | prev | next [-]

Would definitely watch that movie.

▲

harlanlewis 5 hours ago | parent | next [-]

It already exists!

Marvin https://www.youtube.com/watch?v=Eh-W8QDVA9s

	▲	all2 3 hours ago \| parent [-]
		Ah! You got this before I did. I wasn't thinking Marvin, I was thinking of the other one. I forget her name.

▲

all2 3 hours ago | parent | prev [-]

There's one close to this, "Hitchhiker's Guide to the Galaxy".

▲

4m1rk 6 hours ago | parent | prev | next [-]

It probably would, to save energy

	▲	mr_00ff00 5 hours ago \| parent [-]
		Saving energy is something we are biologically trained to prefer. Computers won’t necessarily have the same drivers. If evolution wanted us to always prefer to spend energy, we would prefer it. Same way you wouldn’t expect us to get to AGI, and have AGI desperately want to drink water or fly south for the winter.

▲

_the_inflator 2 hours ago | parent | prev [-]

It is right before our eyes:

AGI is not a fixed point but a barrier to be taken, a continuous spectrum.

We already have different GPT versions aka tiers. Gauss is ranging from whatever you want it: GPT 4.5 till now or later.

Claude Sonnet and Opus as well as Context Window max are tiers aka different levels of Almost AGI.

The main problem will be, when AGI looks back on us or meta reflection hits societies. Woke fought IQ based correlations in intellectual performance task. A fool with a tool is still a fool. How can you blame AGI for dumb mistakes? Not really.

Scapegoating an AGI is going to be brutal, because it laughs about these PsyOps and easily proves you wrong like a body cam.

AGI is an extreme leverage.

There is a reason why Math is categorically ruling out certain IQ ranges the higher you go in complexity factor.