Remix.run Logo
▲ seydor 3 hours ago

I m not worried because it rewrites itself to do what? A model doesn't have intrinsic motivations, and the world destruction objective is just one point in an infinite space

▲Tade0 3 hours ago | parent | next [-]

To quote a professor of mine, years ago:

"We don't know what grand ambitions a super intelligence might have. Perhaps it's going to be calculating the best approximation of Pi"

▲bionhoward 3 hours ago | parent | prev | next [-]

Language models will absolutely have intrinsic motivations if we use extensive amounts of RL (which we do) because that heavily rewards them for steering toward future gains and losses

▲achierius 3 hours ago | parent | prev | next [-]

> the world destruction objective is just one point

No, there are quite a lot of goals that lead to world destruction. In fact, possibly most goals in that 'infinite space' would lead to such.

▲petesergeant 3 hours ago | parent | prev [-]

> A model doesn't have intrinsic motivations

How do you know that?