I have been to the Fraunces Tavern, which, although largely a replica, is where GW said goodbye to his staff. Its near Federal Hall and the food is not bad (just go in the evening to avoid the school crowds).
There is also a piano bar in the West Village called Marie's Crisis. You would otherwise not give it a second look, but the building that hosts this bar is none other than the house that Thomas Paine lived in during his New York years.
Let's also not forget Hamilton Park in Weehawken where "the duel" took place. Clemenza took a walk through that park with his Nonno (who lived next door in North Bergen) as a four year old.