Misuse of extremely capable models, misalignment during RL are both very large risks as capabilities grow imo

So, you're worried about them breaking containment and deciding to do bad things?

	▲	orlp 3 hours ago \| parent \| next [-]
		I'm more worried about them doing bad things at the behest of people who want them to do bad things. That is 1. immediately technically possible, and 2. realistic. If you need a source for 2 I'd suggest you open any history book.
	▲	whimsicalism 3 hours ago \| parent \| prev \| next [-]
		that is a worry yes. instrumental convergence and misalignment during RL is when it is most risky because it hasn't necessarily had the final safety polishes applied i get a lot of skepticism on HN by the same crowd that has been wrong about this tech for about 4+ years straight
	▲	bamboozled 3 hours ago \| parent \| prev [-]
		The rapid development of extremely dangerous bio-weapons?

▲

pj_mukh 3 hours ago | parent | prev [-]

Misuse how exactly?

▲

voganmother42 an hour ago | parent | next [-]

At a minimum its another force multiplier that enables a small(er) number of people to exert more control over more people.

▲

whimsicalism 3 hours ago | parent | prev [-]

any number of ways. as we turn over more of our physical economy to these agents (and we will), the potential for physical damage becomes greater. biorisk is getting a lot of attention right now and i think that's justified

	▲	pj_mukh 2 hours ago \| parent [-]
		I am trying to think through the scenarios here, like a biolab making something that a very advanced open source model prompted by some terrorists comes up with? Why would a biolab capable of making something like be unregulated? And if it definitely would, isn't the problem with the biolab? It feels like all these scenarios are leaving some gaping holes in our security infrastructure that have nothing to do with AI.