If you steal a book and read it, should you have to pay every time you use the knowledge gained or recall parts of it from memory?

No. People are not LLMs. And even if some argue that they are mechanically similar, they are legally distinct.

If I charged people for the privilege of listening to me recite relevant parts of the book to them for profit? Yes. Depending on the copyright.

If I perform a song in public then yes, I should pay the creator every time I play it. I fail to see the difference here.

What if you steal a CD and then play it on your radio station each morning?