Not sure why I can't respond to your other comment. But this is in response to your comment:

> they should lose access to the pool, not their house.

To enforce this you have to be able to gate, secure, and prevent them and their guests from using this. That costs a lot more money just to ensure they pay in. The HOA is the least worse option to "consent" and ensure that they do. Taking away the house is a much later step after a letter, 2nd and 3rd warnings, fines, ignored fines+interest, and a lein is ignored.

Every HOA-ran pool I’m aware of has a gate with keys or key fobs to prevent non-residents from access. From a liability perspective this system would be a requirement in many areas unless only open with a lifeguard present.