The problem I'm having with agents is not the lack of a knowledge base. It's having agents follow them reliably.

This matches my experience. The bottleneck isn't what the agent knows, it's what the agent can verify. A knowledge base tells it "don't do X", but the agent still has to remember to check. Giving it a tool that returns ground truth works better. The agent calls the tool, gets a concrete answer, acts on it. No memory required, no drift over time.