"Where do we draw the line?"

There will be no line as long as there is the rush to win the capitalist game.

UNTIL -> The ball of neurons begins outthinking the humans. Probably also fused with some AI augmentation.

It only takes a few percentage points for a Human to outthink a Chimp. This new 'thing' will dominate the humans.

This is where I'm at as well. I don't think we'll see true AGI until we go beyond silicon. It can't grow on it's own, and we'd burn the world down trying to get it to scale.

A living bundle of neurons that can grow and learn is exciting to think about.

It's also terrifying to imagine the ramifications considering how things are going with silicon based AI.

Username checks out.