When I give an estimate, I always reiterate that it's just an estimate that is based on limited and incomplete information, and that the real number can be more or less. If they don't like it, they're free to put someone else on the project.