> Open-source only matters if you have the time/skill/willingness to download said source (and any dependencies') and compile it.

Not really. The fact that an application is open-source means its originator can't rug-pull its users at some random future date (as so often happens with closed-source programs). End users don't need to compile the source for that to be true.

> Otherwise you're still running a random binary and there's no telling whether the source is malicious or whether the binary was even built with the published source.

This is also not true in general. Most open-source programs are available from an established URL, for example a Github archive with an appropriate track record. And the risks of downloading and running a closed-source app are much the same.