One reason for that might be the size of the baby homo sapiens scull/head upon birth. Bigger brain might have meant female Neanderthals couldn’t give birth to Homo Sapiens babies. Just a theory.

Neanderthals had larger cranial capacity than modern humans.