[flagged]

wizzwizz4 5 hours ago | parent | next [-]

> Try some context engineering. Try customizing your harness. Try having the harness improve itself. These are AI 101 lessons

If AI is really that complicated, it sounds like it would be easier to write an ordinary computer program to aggregate job boards.

▲

ndriscoll 5 hours ago | parent | next [-]

Yes, one of the obvious ways to me to use these models is to tell them to write such a program. It can then go figure out data extraction and normalization. This is "the harness improving itself". Have it write tools to do its tasks.

▲

root-parent 5 hours ago | parent | next [-]

I think you missing the problem with composition itself. The tool generation is the easy part. Knowing which tool is needed, specifying it correctly, validating it against current context, knowing why it failed, and deciding when it needs replacement are separate tasks, and they compose, and composition compounds and fails.

	▲	ndriscoll 2 hours ago \| parent [-]
		I've never found that to be a problem (using codex since the beginning of the year and having it make dozens of shell scripts for itself). It adds usage/help without me ever having had to ask, and if the tool has an issue, I've found it will generally debug and fix it on its own.

▲

wizzwizz4 5 hours ago | parent | prev [-]

I may not be personally familiar with the latest and greatest moving target in agentic AI, but I am familiar with AI-generated "slop code". What I see is consistently defective, to the point where I would not trust an AI-generated data pipeline to produce remotely accurate results, even though messing up a basic data pipeline that much is a difficult task for humans. (Then there's the tendency for an LLM post-processing the tool output to smooth over such flaws, massaging or fabricating conspicuously missing or corrupt data to hide what would normally be extremely obvious warning signs.) If it is, as the proponents claim, even possible to entice these systems to produce decent programs (other than by direct plagiarism), it must require skill way beyond that of the average prompter with 18 months' experience to explain my observations.

When I say "write an ordinary computer program". I mean just writing the program, in a programming language. Your comment expresses disagreement with me, so should have started with "No", not "Yes".

▲

criley2 5 hours ago | parent | prev [-]

I totally agree - designing a competent AI agent with a fully customized harness to successfully pull off this task is a much more challenging engineering effort than merely creating an ordinary computer program. Had OP made chatgpt write an ordinary program instead, they likely would have succeeded in their task.

▲

root-parent 4 hours ago | parent [-]

You said "Had OP made ChatGPT write an ordinary program" but that assumes enough structure exists to specify that program. If so that may be the right architecture, but also demonstrates why the "agentic AI" does not automatically solve the original open ended task.

You have converted a fuzzy task into a conventional software engineering problem, and then relying on conventional software for the reliability :-)

	▲	fn-mote 4 hours ago \| parent [-]
		> converted a fuzzy task into a conventional software engineering problem, and then relying on conventional software for the reliability What can I say? This is how I get results from AI. It also gives me context & tools to fight the AI when I have to.

▲

tcp_handshaker 5 hours ago | parent | prev [-]

Its the tools. Sell those RSUs while they last.

Most of these are 2026....

Frontier LLMs Still Struggle with Simple Reasoning Tasks - https://arxiv.org/abs/2507.07313

General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks - https://arxiv.org/abs/2604.11778

LLMEval-Logic: A Solver-Verified Chinese Benchmark for Logical Reasoning of LLMs with Adversarial Hardening - https://arxiv.org/abs/2605.19597

LogicGraph: Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification - https://arxiv.org/abs/2602.21044

Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models - https://arxiv.org/abs/2607.08317

Vision-Language Models Lag Human Performance on Physical Dynamics and Intent Reasoning - https://arxiv.org/abs/2601.01547

Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects? - https://arxiv.org/abs/2605.20448

The Reversal Curse: LLMs Trained on “A is B” Fail to Learn “B is A” - https://arxiv.org/abs/2309.12288

Large Language Model Reasoning Failures - https://arxiv.org/abs/2602.06176

▲

criley2 5 hours ago | parent [-]

What a sloppy reply. You've hijacked a thread on mathematics first to complain that your incompetent attempt to use ChatGPT to find a job failed, but it seems now that this was a ruse to instead begin arguments unrelated to the article at all where you just spam arxiv links you've never read to "prove" that AI is a scam.

This comes across, frankly, as either Dunning-Kruger (classic illusory superiority), or potentially as mental illness. The slop dump is highly reminiscent of how a schizophrenic friend of mine communicates.

Do you really think slopping down a bunch of random arxiv links "proves" that AI is a scam and you're so smart and everyone else isn't?

Most awkwardly for your arxiv slop -- most of this is irrelevant to your central claim, and you've missed papers that are much closer.

For example your LogicGraph paper: "Can't exhaustively enumerate all minimal proofs" is not "can't distinguish Ireland from London".

Or your "Do VLMs Understand 3D Scenes..." is nothing more than citation decoration, completely irrelevant to our discussion.

Or your "Frontier LLMs Still Struggle with Simple Reasoning Tasks" which is potentially your pièce de résistance, it supports brittle multi-step constraint handling, but isn't remotely an eval of a modern web-search agent.

For example, VibeSearchBench would have been far more relevant to your claims https://arxiv.org/html/2605.27882v1 (but still obviously not proof that AI is "a parlour trick")

Going further: my point that we need to discuss your beginner's approach to the harness is substantiated clearly here: https://arxiv.org/html/2605.23950v1

Finally, failure to exhibit human-like generality is not evidence of absence of intelligence. It is evidence that whatever cognitive machinery LLMs possess has a very different error distribution from ours. Your General365, LLMEval-Logic and the Reversal Curse are actually fascinating evidence for that jaggedness, rather than proof of your claim that AI is a scam.

▲

suddenlybananas 5 hours ago | parent [-]

Why are you so upset that someone is criticising LLMs that you call them schizophrenic? (I'd recommend refreshing your memory with this https://news.ycombinator.com/newsguidelines.html )

▲

phantompeace 4 hours ago | parent | next [-]

Nobody called anyone schizophrenic. They noted that the irrelevant link spam in a reply was reminiscent of schizophrenic posting. If you've spent any sort of time on places like 4chan, you'd understand. I don't think posting the HN guidelines is warranted here.

▲

suddenlybananas 3 hours ago | parent [-]

>Nobody called anyone schizophrenic. They noted that the irrelevant link spam in a reply was reminiscent of schizophrenic posting.

Come on, that's being incredibly pedantic. If I say, your comment reminds me of someone being an asshole, I'm just using a circumlocution to call you an asshole. No?

	▲	redsocksfan45 44 minutes ago \| parent [-]
		[dead]

▲

4 hours ago | parent | prev [-]

[deleted]