I do think prompting and reference files (e.g. for architecture, tech stack, …) can be extremely helpful. I also do this in my projects (challenge is keeping drift of these documents at bay).

What I wanted to emphasize is that whatever is in the context (whether system prompt or user message does 'steer' the model in a strong way, so everything in the context affects overall performance in a way. Even if it is 'just' net neutral it takes space up in the context window.

The context window is very very precious, everything that goes into it should help (not just hopefully help).

The challenge is coming up with good stuff to put into that context. A good agents.md file will be better context than whatever the popular harnesses have in their system prompt.

Also good to keep in mind that newer models are very good and more agentic than older models so they are better at exploring their environment based on the tasks you give them.