I also do not see any requirement in the quoted text that the casted pointer be dereferenced before noting "the behavior is undefined".

In practice performing a cast doesn't really do much until you dereference, but without a carve out in the spec, it does really mean "the behavior is undefined".