Inverse is still 0.0 technically, but yes, there is a trick you can use with Inf and SIMD to mask them out, so Inf is sometimes used.

However, I'd just condition it for the moment.

so:

invDirX = dirX != 0.0 ? 1.0 / dirX : 0.0; etc, etc for each dimension.

Obviously doing the != 0.0 comparison is not great, as it suffers from potential issues again (especially if you have denormals), but you can generally get away with it I've found in most cases.