So what happens when the world becomes hyper optimized with closed loop AI agents recursively trying to optimize everything deemed sub optimal?

▲

mofeien 7 hours ago | parent | next [-]

I would assume that shortly after, the solar system will be hyper optimized as well, then the milky way, then the local cluster, and so on. Everything will be close to optimal afterwords, and I sure hope we will have specified the target function for that optimization correctly in the single attempt that we will have had.

	▲	Readerium 5 hours ago \| parent [-]
		Loll

▲

peheje 7 hours ago | parent | prev | next [-]

there will be a lot of paper clips

▲

simianwords 5 hours ago | parent [-]

Often repeated meme doesn’t have any bearing to reality.

The orthogonality thesis sounds like a fun gotcha but if you give it some thought you realise how strange it sounds and the opposite thesis - collinearity thesis is actually correct.

1. Intelligence transfers and compounds

2. Goals of agents are not arbitrary

3. Our goals and agent goals are more likely to be aligned at the deeper level

	▲	IsTom 4 hours ago \| parent [-]
		When primate family produced a super-primate intelligence it sure aligned with the good of all of them.

▲

Groxx 5 hours ago | parent | prev | next [-]

Github outages will probably get worse.

▲

layer8 4 hours ago | parent | prev | next [-]

If it optimizes itself away because it’s suboptimal, that wouldn’t be the worst outcome. ;)

▲

7 hours ago | parent | prev | next [-]

[deleted]

▲

4 hours ago | parent | prev [-]

[deleted]