I believe with direnv or a similar tool (e.g. Emacs’s directory-local feature) one can append $REPO/.config to XDG_CONFIG_HOME, $REPO/.local/bin to PATH and so on so that when in the context of a particular directory everything Just Works.

I think all this agentic stuff could live quite happily in $REPO/.config/agents/.