"The fact that neanderthals disappear in Europe, about the same time as modern Homo sapiens appear, could be used as evidence to support the replacement hypothesis. However, not all the evidence, however, supports the Replacement Hypothesis in Europe. In 1998 at the Lagar Velho site in Portugal, the remains of a child, aged 3-1/2 to 5 years was found that apparently shows a "mixing" of Neanderthal and Modern human traits. The skull of this specimen has a pronounced chin, which is a hallmark of modern homo, however, post-cranially, the boy's bones were quite robust, and he was barrel chested and had short lower limbs, all of which are cold adaptations which characterized Neanderthal. "From here.
If we did not, in fact, interbreed with them, it might have been a chromosomal incompatability (the evidence is the last common ancestor we shared with them lived between 500,000 and 850,000 years ago) or an anatomic issue.
It's worth noting that the Lagar Velho fossil and what it signifies has been the subject of acrimonious scientific debate.
The Lagar Velho child has subsequently proven to be a modern human child not a hybrid.
Modern Human Neanderthal Hybrid