> Didn't someone say LLMs memorize Harry Potter books? You can't have it both ways.

What both ways? You can't consistently get it to output Harry Potter verbatim. That's my point - not reproducible.