Perhaps encoding such things in comments at all is the wrong approach? E.g. If my linter misbehaves, why can't I right click and ignore the red line in the IDE instead of encoding it into my source file.

Encoding it into your source file has positive externalities. If you're using source control, the comment controlling the linter is tracked alongside the rest of your code. You can track who added it and why. You can share this comment with other engineers on your team.

You could also imagine other representations of the same data (e.g. one large file which includes annotations, or a binary representation). But in this case you lose colocation with the code they're annotating, so it's more likely to drift.

I fully agree that there are probably better UX's out there for this flow, but the source annotation approach is popular for very good reasons.