Ah right, I've forgotten because I'm using a multi-user strategy and a patched version of the runner at this point anyway. The config directory for each runner is normally based on its install path (insane), something like that?

The config directory based on the install directory, yes.

To be fair, running multiple copies of a service isn't that common.

E.g. if you wanted to run multiple copies of PostgreSQL you can't just do a vanilla install.