What do you do with the spent material? Can it be recycled? (i.e., replenished through a 'breeder' or something like that) If not, you've still got a serious problem of waste material disposal. We really don't have any 'safe' way to store material that maintains radioactive toxicity for tens of thousands of years.
Personally, I think the only real solution is to launch the stuff into space and park it in a stable orbit around the sun. (not in orbit around the earth) At some point, if mankind survives as a species and the muslims don't drag us back to the seventh century, I'm sure we'd be able to do something with this stuff. If not, fire it into the sun, and let it be our solar garbage disposal.
It takes more energy to put something into the Sun (meaning force it to slow down, so that it falls into the Sun), than it would take to make an object leave the Solar System, as the gravity of gas giants Jupiter and/or Saturn could be used to accelerate it.
Plutonium leftovers can be used for further nuclear fission. That is how France uses their nuclear waste.
Actually, we have several.