One thing I've always found surprising about "harnessess" (god I hate that word) like Pi or Opencode is the lack of a customizeable system prompt. I can understand it for closed source ones, but open ones?

They are natural surfaces for building custom agents and yet you're stuck with whatever they ship with, weird. It's not like it's too complicated api-wise either.

There must be something I ignore.

▲

simonw 6 hours ago | parent | next [-]

Pretty much everything in Pi is handled by extensions. There's an extension for customizing the system prompt here: https://pi.dev/packages/pi-custom-system-prompt

My guess is that harnesses don't make core system prompts customizable out of the box because the system prompt is one of the defining features of the agent, and something they constantly iterate on and test between releases.

Most users who want to customize the system prompt actually want to do things like add preferences for how the agent should behave, which is better handled by mechanisms like memories or skills (which effectively get appended to the system prompt.)

▲

epolanski 5 hours ago | parent [-]

Oh no, memories and skills are terrible replacements for system prompts.

Not only they get "lost" and ignored as the context grows, but the baseline behaviour of system prompts is retained in the agent.

▲

Supermancho 5 hours ago | parent [-]

Memories are implemented differently agent to agent. They are usually implemented by yet-another-model-call, as a distillation of typed prompts (working memory). Conversational communication is messy with a lower signal to noise ratio than the distillation (semantic memory). Semantic memories are much better than appending raw historical prompts.

Skills are prompts, albeit in a specific format. This is apparent in say, Codex where $MYSKILL is literally injecting the skill-prompt inline into a typed prompt. This all gets passed into the semantic memory system anyways, refining away cruft like redundancy, pleasantries, et al.

	▲	epolanski 2 hours ago \| parent [-]
		And none of this works properly. None. You have to remind it what's in it's own memory, or the subagent skill is influenced by the main system prompt. It's sloppy vibe coders productivity porn.

▲

solarkraft 6 hours ago | parent | prev | next [-]

I agree. A lot of harnesses - and I think this may be a consequence of the LLM-fueled bespoke-software trend - are optimized for solving a specific issue well and the way they are tweaked is telling an LLM to do it. This resolves the need for natural extension points.

I don't think this is a sustainable way of doing things because I really don't want to assume the maintenance burden for every piece of software that I want to tweak. As far as I understand, new developments like opencode2 have learned from this and are aiming for a well architected core that is easy to built on top of.

▲

qiine 6 hours ago | parent | prev | next [-]

In pi you can replace it with ~/.pi/agent/SYSTEM.md but its largely procedurally generated so you have to do a lot more than simply writing a different markdown file for it to be worth it in my experience.

▲

kzrdude 5 hours ago | parent | prev | next [-]

I don't get it, for pi.dev it's easy to replace or append to the system prompt using a text file. Per user or per project.

▲

slickytail 6 hours ago | parent | prev [-]

[dead]