I find if you figure out the right unit boundaries, and find a good way of testing the code, you can often keep the tests around long-term, and they'll be very stable. Even when you update the code you're testing, if the tests are well-written, updating the tests is often just a case of running a find-and-replace job.

That said, I think it takes a real knack to figure out the right sort of tests, and it sometimes takes me a couple of attempts to get it right. In that case, being willing to delete or completely rewrite tests that just aren't being useful is important!