Is copying and pasting someone else’s words and publishing it on your website or personal profile not First Amendment protected speech?

You’re quoting someone else’s speech. The quotation is your speech, but the literal words were not.

I’m actually fine with the idea that people are quoting LLMs but there’s this conflation I find many people where they treat the LLMs output as their own literal thoughts.

The language around this distinction is clunky currently so if you meant the former point I agree, but I had interpreted the comment I was responding to as the latter.