What did Panetta do when he was with the CIA? Defense or CIA, he should stay in DC--those positions are too important. Better for his family to visit him.
It's not like he'd be stuck in a Motel 6.