He's got my vote.
I have some problems with Jackson on the twenty, but he opposed a central bank with all his being. This led to 80 years of stability in our currency that has not been seen since over the past 100 years. But the biggest one, when he left office, he had accomplished a feat that had never been done before and has not happened since. The United States had no national debt. It was paid in full.
However the Indian Removal and the Trail of Tears is to me the major stain on his legacy. So if we do remove him from the twenty dollar bill, I would humbly suggest that the most appropriate person to replace him would be Shawnee chief Tecumseh.
He's got my vote.