As someone who frequently says “don’t connect these $things” to the Internet, I appreciate the boost.

Half my compute vendors are raising prices because of this insanity.

Or make them part of the consequences. Give them skin in the game. “Let’s not use AI” is dumb and impossible

Maybe the best move is to not play the game.

This is the path Apple has taken.

But the best possible move is to make money from it. Short the "Magnificent 7" stocks --- buy "SQQQ" ETF --- when the time is *right*.

> They have no real world experience. They feel no real world consequences. They have no real stake in any decision they make.

Why do you let politicians do any kind of decision making?

They have no real stake in any decision they make.

And they are not human. Not even a sociopath or psychopathic human. At best they might be able to estimate casualties. LLM's probably can't even reach the logic conclusion of the fictional WOPR Joshua from the movie Wargames [1].

Make LLM's win every game of tic-tac-toe and see if it reaches the same conclusion of WOPR. [1]

...

Edit: (Answering my own question) From Gemini:

Yes, many LLMs (GPT-4, Claude 3, Llama 3) have been tested on Tic-Tac-Toe, and they generally perform poorly, often playing at or below the level of random chance. While they can understand the rules, they struggle with spatial reasoning, often trying to place a piece in an occupied spot, forgetting to block opponents, or failing to win.

If LLM's can't even figure out tic-tac-toe then surely do not give these things the ability to launch any kind of weapon. Not even rubber bands.

[1] - https://www.youtube.com/watch?v=s93KC4AGKnY [video][6m][tic-tac-toe]

> People in the world have limited experience about war.

Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.

> And now we are at a situation where nuclear escalation has already started (New START was not extended).

This is a massive understatement. Russia has announced, and probably tested, https://en.wikipedia.org/wiki/9M730_Burevestnik . This is basically Project Pluto reloaded, but now as a Russian instead of a US missile.

I remember reading about Project Pluto some 25 years ago or so. It was terrifying to read about. And now Russia has realized it.

> People in the world have limited experience about war.

Most (but not all) people have empathy, which allows them to understand the harm of their actions even without direct experience.

I don't think I will ever trust that any AI has empathy even if it gives off signals that it does.

I only trust that it exists in people because of my shared experience with their biology.

I swear I'm not trying to start a flame war, but I think it'd be useful/valuable to know where you're from and what country you live in, as this certainly shapes how we feel about these sort of issues.

I've also been dabbled in such thought experiments with friends lately, and so far we've all landed at very different conclusions, even thought there are some reasons that it might make strategic sense at the moment.

What happens in rimworld, stays in rimworld?

if you win the war then there really isn't any such thing as a war crime. Worst case is you feel guilty about it, there aren't any other consequences of your actions.

It holds up if you assume war crimes are beneficial to your goals but there is quite a lot of evidence, and sophisticated theory going back to clausewitz, that they mostly aren't.

They can look useful at a certain level of conflict, but once you are thinking of war as being a tool for accomplishing policy goals (how modern nationstates view it), a lot of the things you would "want" to do stop being useful.

Wars that can be won quickly through decisive military action alone are quite rare historically! More often things like support/enmity of the local population, political will in the home state, support for recruiting or tolerance of conscription, influence of returning (whole, dead, injured, all) veterans on the social structure all become more decisive factors the longer a conflict runs.

[dead]

AI has zero understanding of reality. It just regurgitates what it is told from training. There is no feedback loop to learn nor any consequence to the reasoned results.

Us human hallucinate, daily in fact. Example for people that have never had long hair.

1) Grow your hair long.

2) Your peripheral vision will start to be consumed by your hair.

3) Your hair will fall and sway causing your brain to think in flight / fight mode and you will turn your head to see.

4) Turning and looking causes feedback to acknowledge it was an hallucination.

5) Your brain now restricts the flight / fight mode because it was trained with continual feedback that it was just the wind blowing it or your head's juxtaposition that caused it.

Even though I told you about this and it is the first time growing your hair after, your brain still needs the real world experience to mitigate the hallucination.

AI has none of these abilities ...

Almost no human has real world experience of the consequences of nuclear weapons.

Exactly!

Humans possess this amazing ability to understand and extrapolate beyond personal experience.

It's called "intelligence".

AI is not at all like real intelligence. Computers do not know what words mean because they do not experience the world as we do. They don't have the common sense or wisdom that people accumulate through the experience of life. Humans can understand the consequences of nuclear war. Computers can only predict the next best word in their response from a statistical map that has no connection to meatspace.

Speaking of Iraq, Sadam decided against using chemical weapons in the Gulf War because he had received intelligence from the Russians that the Americans would counter with nuclear weapons and he didn't want to risk that.

They are simple next word predictors. Wether they recommend a nuclear strike solely depends if that was present in the training texts.

What's being revealed is "Nuke 'em" is an optimal strategy for the goal.

You're giving AI way too much credit.

Most likely, AI really didn't optimize anything.

It most likely engaged in a probability driven selection process that inevitably lead to the most powerful weapon available.

Change the goal, change the result.

Yes. The tricky part is recognizing the need to change the goal.

Achieving this implies you already have an answer in mind that you want to lead AI toward. And AI is often happy to accommodate --- because it is oblivious to any consequences.

Someone's getting nervous about being replaced by AI

Are you an AI? Because your conclusion may seem obvious enough but suffers from lack of input.

I run my own company so I can't be replaced by AI. And I do look forward to competing against AI converts in the marketplace.

In any case, we're extremely lucky that there's about 0.001% probability of LLMs being a path to AGI.

It's pretty safe to say that AGI requires a lot more than picking plausible words using probability.

The danger is the number of people in positions of leadership who don't get this. People who are easily seduced by the "fake intelligence" of LLMs.