You could bury them....Wait...
You actually could bury a print out of the keys and then delete the wallet info.
5 Security Tips for Bitcoin Beginners