The thing about this kind of 'just so' story causal mechanism is that we still have to actually do the science to find out. Your body does filter and kill potentially cancerous cells all the time already. And cancer cells aren't like, some super thing that evolved to kill you specifically. My just-so story goes like this: 'the cancer cells die because they're suddenly outside of the specific bodypart that they were exploiting'. And we're probably both right, depending on the location of the cancer, the type of cancer, etc.