If these different agents could agree on a standard location that would be great. The specs are almost the same for .github and Claude but Claude won't even look at the .github location.

There already is, it's ~/.agents and you use symlinks for .claude, and the dir structure is pretty similar and anything you want to reuse across models is pretty standardized, just not formalized.