> Maybe one day agents will have their own experiences, too. Maybe they'll develop intuitions they can't quite explain, just like we do.

Machine / program / software cannot have these things in my opinion because this stuff is produced by human consciousness which we cannot even fully explain yet. Someone has "to teach" the agent to have this stuff, but we don't have that knowledge. And also "intuition" is a feeling (like love, hate, sadness, joy) so no agent can have that until they have the limbic system, prefrontal cortex and other stuff.

Not everyone knows how the brain works and many believe it to be one organ.

"We can't explain it therefore we can't create it" doesn't seem like a strong argument, emergent behavior and whatnot.

Hell, the fact we don't know what consciousness is means that some LLM systems (or trees! or planets!) might already have it, and we'd be none the wiser. Not that I believe this, but as these systems get larger and more sophisticated it's an increasingly concerning proposition that we have no way to validate either way.