It would be relatively easy to scan the output of the LLM for copyrighted material, before handing it to the user.

(I say 'relatively easy'. Not that it would be trivial.)