instead of using ~/bin I use ~/installed/bin, sometimes I need build a command from source then install it, which might have share/ man/ etc so I can avoid installing them under the home dir.

~/.local/ could work for this, and as another user mentioned easier to separate the back up.