If a legit user accesses the link through an <img> tag, the browser will send some telling headers. Accept: image/..., Sec-Fetch-Dest: image, etc.

You can also ignore requests with cross-origin referrers. Most LLM crawlers set the Referer header to a URL in the same origin. Any other origin should be treated as an attempted CSRF.

These refinements will probably go a long way toward reducing unintended side effects.