in my book anything that is (I'm sure well intentioned) and injected to help the agent — but doesn't help it — is a waste of tokens

but even injected context that when I read it sounds useful can oversteer the model and make it second guess or take a more complicated route than it normally would

(you can see this when looking at traces with and without that injected context)

often harnesses also mention in their system prompt locations of markdown files that the model can consult if the model thinks they might help

that hint alone as part of the system prompt can be strong enough to make the model read in more tokens than would have been necessary

'spam' is maybe a harsh way to say it

unfortunately I don't see an easy way other than to invest time and tokens into finding out which parts of the added context (in system prompt, injected in turns etc etc) are actually helpful or harmful and when

I'm just doing the easiest thing I could think of: start from nothing or close to nothing

that seems to work better than what most harnesses are doing

turns out GPT 5.6 Sol is all you need

▲

kelnos a day ago | parent | next [-]

> in my book anything that is (I'm sure well intentioned) and injected to help the agent — but doesn't help it — is a waste of tokens

How do you tell, though? I guess what I'm asking is: the data you presented shows some crazy differences, but the token burn alone doesn't tell us enough. What was the output of the task like? Did the harnesses that burned fewer tokens give you as good a result as the ones that burned more?

I guess it's subjective, of course, but nearly everything about LLM use is...

▲

tosh 17 hours ago | parent [-]

I agree, only looking at the token burn is not enough

in this case it was 10 tasks and all harnesses could complete the tasks successfully, of course now the question is: will this hold for more and more complex tasks but I had to start somewhere :)

Checksum: Compute a file’s SHA-256 checksum and save the exact digest to an output file.

Log correlation: Correlate nested service logs to identify and summarize a request’s complete execution path.

CSV report: Parse quoted CSV data and aggregate paid orders and exact decimal totals by region.

JSONL join: Join related JSON Lines datasets and produce a correctly grouped and ordered report.

Archive repair: Find the correct version of a corrupted file in a tar archive and restore it.

SQLite migration: Safely migrate a SQLite database schema and verify the resulting data and constraints.

Python bug fix: Repair an interval-merging implementation so it passes visible and hidden edge-case tests.

Python CLI: Implement a robust command-line program that reads JSONL and reports validated statistics.

Multi-file feature: Add an atomic feature across a small Python package, CLI, and associated tests.

Pipeline repair: Fix a Make, shell, and Python reporting pipeline so it handles general input and passes verification.

	▲	DonHopkins 16 hours ago \| parent [-]
		[dead]

▲

mikodin a day ago | parent | prev | next [-]

I am super curious how much in your opinion the extra prompting is useful?

>often harnesses also mention in their system prompt locations of markdown files that the model can consult if the model thinks they might help >that hint alone as part of the system prompt can be strong enough to make the model read in more tokens than would have been necessary

I purposefully do this as I imagine it is useful. In my project I am seeing solid adherence to norms and a deep capacity to iterate on completed features. Essentially for each feature I have the model make a {featureName}.AGENTS.md at the root folder of the feature, where it maintains what is going on.

I am moving between Claude Code and Codex atm, but I began this pattern when pi + kimi 2.6 was my main driver.

	▲	tosh 17 hours ago \| parent \| next [-]
		I do think prompting and reference files (e.g. for architecture, tech stack, …) can be extremely helpful. I also do this in my projects (challenge is keeping drift of these documents at bay). What I wanted to emphasize is that whatever is in the context (whether system prompt or user message does 'steer' the model in a strong way, so everything in the context affects overall performance in a way. Even if it is 'just' net neutral it takes space up in the context window. The context window is very very precious, everything that goes into it should help (not just hopefully help). The challenge is coming up with good stuff to put into that context. A good agents.md file will be better context than whatever the popular harnesses have in their system prompt. Also good to keep in mind that newer models are very good and more agentic than older models so they are better at exploring their environment based on the tasks you give them.
	▲	cyanydeez a day ago \| parent \| prev [-]
		I tailored an opencode agent with terse, minimum instruction, going as far as using ascii symbols and definitions. Works fine. My conclusion is treat it like a starting point and not a holy bible of everything the model needs to know. I pair this with making sure task files are commented with headers like build scripts so context+instruction is consistent. The happy path then is the LLM pulling in context where it needs it.

▲

trollbridge a day ago | parent | prev | next [-]

To be fair, 5.6 Sol is pretty much doing a ton of reasoning you can't see (and which you don't think about the cost of if you're using it via an OpenAI Pro subscription)

▲

16 hours ago | parent | prev [-]

[deleted]