I'm gonna cite myself here:

«We came full circle on the problem that also exists with Google Search, and why so many people don't get good results out of it.» [0] It is not that access to all information was hidden to anyone before, it was harder to find it, sure. Now it can be found easily and aggregated, however, it is not as simple as saying "Do X" and believe that X is correct. Recall the infamous Swing & Product Management meme? [1] The problem is the absence of information you provide, the LLM will have to make a guess of what you mean, much like the human. Likelihoods of this guess being correct depends on the domain and background. So yes, it'll likely build whatever you ask it (I do think we're there already), but if that matches what you intended to actually get is vastly different. You'll be settling with "Good enough" though and declare it working.

Others already pointed out that 6 months seem to be the constant of distance, irrespective of the date.

[0] - https://news.ycombinator.com/item?id=50063427#50063744

[1] - https://www.productftw.com/productftw-2-the-tire-swing-carto...

I'm done waiting 6 months. Just let me know the night before when I don't have to come in to work tomorrow.

[dead]

It's funny you mention this, because I've been thinking lately that all the focus on code quality benchmarks for LLMs seems to miss something. In my experience, LLMs aren't actually all that good at prompting other LLMs, and that's exactly the skill that matters for proper subagent usage.

The corpus of code available to train LLMs on is rich in both volume and quality. The same can't be said for LLM prompts. Prompts only came into existence alongside LLMs themselves, so most of the prompts out there are actually LLM generated, and training on them causes every model to overfit to the same patterns.

LLMs can give clear and concise directives to other LLMs, but only for basic tasks. Beyond that, they frequently overstuff their prompts with too much prose and too much direction, and worse, they often fill them with subtle contradictions.

Of course, there's a spectrum here. Some models turn out to be better at this than others, and I suspect this is what differentiates Fable-type models from the rest, and why vendors have started keeping their models' reasoning confidential.

That's what I've been doing in areas I lack knowledge tbh. Instead of a half assed attempt at describing what I need, I ask an llm to take my base idea and write it up using industry standard jargon as if I were commissioning a team to do the work. Has been working pretty well so far!

Okay so I just asked it "What question should I be asking you but I don't because I'm not smart enough to think of it?" Response was kind of interesting I have to say.

Healthcare is more than having medical knowledge.

Ten years ago, in France, two doctors (one in a professional setting and the other during a private consultation) asked me about a wound I had on my shoulder; obviously I couldn't identify what it was, and they offered no suggestions, so I supposed it was not something important.

It took a visit to a cardiologist—for an entirely different reason—for him to suggest that I rapidly consult a dermatologist.

It was a carcinoma, and surgery removed it. If ChatGPT had existed back then, this carcinoma would have been removed several years earlier.