Although, I have known using an LLM to do your prior research to itself be a trap.

The searches it runs, and the summaries it provides, are all incredibly sensitive to your choice of words. Words you chose from a state of minimal knowledge. So it’s like a particularly perverse version of the anchoring bias: information that could have led you to a better solution is often actively filtered out of the agent’s response precisely because it leads down a different path from your first idea.

In short, if you ask an agent what’s the best hammer for driving screws, it’s liable not to mention that screwdrivers exist.

▲

estetlinus 4 hours ago | parent | next [-]

Beautifully written. Hammers for driving screws + sycophancy are horrible together.

▲

user43928 3 hours ago | parent | prev | next [-]

Is it?

As in, are you sure, and can you provide concrete examples?

	▲	mylies43 2 hours ago \| parent \| next [-]
		I just tried it with chatGPT asking for whats the best hammer to drive in a nail, and it gave me a list of hammer suggestions :p . I drove it a bit further and specified wood screws and while it did point out a screw driver would be better it still kept recommending more specific hammers for driving the nails instead!
	▲	bunderbunder an hour ago \| parent \| prev [-]
		For example I saw the agent advise someone to introduce a soft delete mechanism into their database schema, thus incurring all the query performance, correctness and maintenance headaches that come with it, even though they didn’t actually use soft deletes to support any product feature. (And indeed the change had plenty of latent defects because the soft delete pattern hadn’t even been correctly implemented.) We talked about it, and it sounds like he had just uasked, “Should I include soft deletes?” And it just gave him a generic trendsloppy answer about how soft deletes are often considered a best practice. He was more of a full stack developer with relatively shallow database knowledge, so he couldn’t see that this was at best half an answer. If he had gone to Google instead, though (and perhaps also scrolled passed the LLM slop that clutters the top Google results these days), he would have had a decent chance of finding a more nuanced article that could have helped him build a deeper understanding of the tradeoffs so that he could make a more informed decision.

▲

cruffle_duffle 2 hours ago | parent | prev | next [-]

Depending on what I’m doing I’ll dedicate a few deep research sessions to building a framework. It will generate some grounding docs that go into the repo and get consumed as we go. Said docs establish terminology, widely known formulas and methods, etc.

And yes the output of these researchers are highly sensitive to prompting. Left to their own devices the LLM will often ship some very biased prompts to its deep research agents loaded with pre-conceived ideas rather than letting the agents uncover things themselves. Then all the agents do is confirm what the prompt told them to rather then “think independently”. (Very similar to open ended interview questions rather than asking yes/no questions)

It’s is far better to spend a session writing writing the research prompt itself.

All of this takes time and tokens of course…

▲

ModernMech 4 hours ago | parent | prev [-]

Yes this is precisely why experts drive LLMs so much better than novices. It’s also why I don’t dismiss pure LLM output as slop and uninteresting; even if we both are using the same LLM, I probably can’t make it output what you can make it output.