Remix.run Logo
▲ RugnirViking 44 minutes ago

> You could potentially ask an LLM to explain it

You can ask frontier models with all the bells, whistles, language servers etc to give you every example of X. And it gives you 12 examples and says that's all of them. When you know damn well there are at least 40 (but not exactly how many). So you say no, you know it has missed some, such as X23, and X27. So it goes away and comes back again and says yes, there are 41 X, here they are. How much digging should you do to see if its right?

I have many times gone looking myself to find that it has still missed some. Asked it to go check for those, to look harder it goes oopsie and says now im sure ive gotten all of them (has it?)

All this to say, I have been burned, repeatedly, multiple times a day, for the last year+. I still use these tools, but I truly cannot understand how some people treat them as oracles that know everything about our codebases