I kind of disagree, but also, I guess, who cares if it’s sci fi or not? Is it a good story that matches your tastes? Then great. The bookstores lump all the “speculative fiction” stuff together, anyway, and I don’t usually look for my next book by genre.

It's great that works for you but there is a reason we have these labels. I've definitely had the experience where I want to escape into some hard sci-fi, and so there's a reason I pick a book from that genre over Anna Karenina. And if someone told me that Anna Karenina was hard sci-fi, it'd color my experience with that book.