If women who were married to people who worked with Feynman had consensual sex with Feynman, what exactly is the problem? It might create a tense work environment, but you should be able separate work from private life.

Bro code I guess.

Or Golden Rule too. I don't think any of us would want to see our wives agree to get banged by some other dude. Even if it was Richard Feynman.