> They could easily say something like "non-atomic misaligned accesses either succeed or trap" or something like that.

That means that the compiler must emit the read, even if the value is already known or never used, as it might trap. There is a reason for the UB!