I don't know a lot about this, but for example an would app side loaded or delivered via F-Droid be subject to this policy?

F-Droid notwithstanding using an alternative app is not a very attractive option right now as there no good alternatives. But if Valve can create an Android app store that competes with the Play store, the in principle situation is very different.

Presumably not, as it's a contractual thing when you upload something to Google Play. Not sure if there are rules about only uploading to Google Play if you use it at all.

I've no idea, but I presume it's not even possible to use the Google Play purchasing API if your app isn't on the Play store.