Okay, I will now say "they are difficult to work with" or "i don't have time" every time a client walks up that I don't want to serve. Infact, that's exactly what a lot of handymen say already to avoid the irritation associated with declining a job. I hope you see how your "ought" will be difficult to implement in practice