Remix.run Logo
georgemcbay 2 days ago

> Yeah, I sometimes use AI for questions like "is it possible to do [x] using library [y] and if so, how?" and have received mostly solid answers.

In my experience most LLMs are going to answer this with some form of "Absolutely!" and then propose a square-peg-into-a-round-hole way to do it that is likely suboptimal vs using a different library that is far more suited to your problem if you didn't guess the right fit library to begin with.

The sycophancy problem is still very real even when the topic is entirely technical.

Gemini is (in my experience) the least likely to lead you astray in these situations but its still a significant problem even there.

jessoteric 2 days ago | parent [-]

IME this has been significantly reduced in newer models like 4.5 Opus and to a lesser extent Sonnet, but agree it's still sort of bad- mainly because the question you're posing is bad.

if you ask a human this the answer can also often be "yes [if we torture the library]", because software development is magic and magic is the realm of imagination.

much better prompt: "is this library designed to solve this problem" or "how can we solve this problem? i am considering using this library to do so, is that realistic?"