This is confused thinking. Why would copyright law care one way or another about whether or not you destroy a book you own?

On the other hand, absolutely yes we should abolish all IP law.

The ruling in the Anthropic case was that their scanning of the books was only fair use because the original physical copy was destroyed. The copyright lobby wanted scarcity so they could monetize things, and now folks are worried about knowledge being scarce. It's a problem of our own making.