I believe the concern is that a failure during launch (think Challenger) might rain down radioactive contamination and debris.
Also one of the concerns when looking for ways to dispose of nuclear fuel waste products.