If they manage to leap ahead they can effectively kill all competition, present and future and you'll be at their mercy.

▲

henry2023 a day ago | parent | next [-]

So the solution is to make the company that can’t avoid sub-AGI models from going rouge to try RSI?

Also, AI performance has proven to be a function of power and compute, and right now most of the compute lies between two companies rushing towards RSI not because they are afraid someone is going to do this before but because they are the ones who wants to build it.

▲

UncleMeat a day ago | parent | prev [-]

If the position is that one of these models will go out of control and destroy humanity then why would having two such models be safer than one?

▲

jacquesm a day ago | parent | next [-]

Given that the arrival of this stuff has been compared to nuclear weapons already the question is whether or not the same mechanisms that kept us from annihilating the planet would hold there too.

I am not sure that it will.

But a balance duopoly tends to be better than a monopoly and if there are multiple poles then so much the better.

I'm past the point where I believe that any of these a-holes is in it 'for the good of humanity', their behavior so far can be extrapolated safely to what they will be like when they are in the control of what effectively would be the largest private arsenal ever created.

For a while I thought the AI doomers were off their meds and too paranoid but I'm coming around to the fact that they may have a point: we are rapidly approaching a point of no return, if we are not already past it and the people in charge are exactly the wrong people to be in charge at such a point in time.

	▲	UncleMeat a day ago \| parent [-]
		How specifically would a balanced duopoly be safer?

▲

bigbadfeline 15 hours ago | parent | prev [-]

> If the position is that one of these models will go out of control and destroy humanity then why would having two such models be safer than one?

Models don't go out of control, they generate totally harmless tokens. The people directing the models go out of control. The owners of the single powerful model are seriously tempted to go rouge and they're going to do it, sooner or later.

Two or more competing groups force the development of defensive capabilities which require strong models too. In addition, offensive capabilities in possession of different competing groups introduce the option of MAD (Mutually Assured Destruction) which is a form of defense that doesn't require defensive capabilities. Worked with nukes.