Honestly, the SCP wiki might scratch this itch for you—it's sci-fi but with a lot of fantasy elements, and I'd put it on the "hard" side of the spectrum. Also, I think Greg Egan's books are pretty out there (the two I've read are Diaspora and Permutation City, whose settings aren't particularly "plausible" IMHO), and they really make you think.

Agreed. It's framing device of reports / scattered documents / etc also remove a lot of the characterization or characters completely and focus just on the "what if?" of the story.