What happens if I wrote a book - sent around to a few publishers and then personally had a change of heart - perhap it has content which I now disavow.

Shouldn't it be within my right to withhold publication?

Or because I refuse to publish, but the publishers have a copy - I should lose the right to control my own work and the copyright falls and the publisher can just publish freely?