To convert between chirality would be an additional complexity hurdle. Like you said, current life generally doesn’t do it at any level of evolution with very minor exceptions.

I am wondering if the concern is that the evolutionary pressure from not having abundant mirror food triggers mutations that make it “ambidextrous”.