Because they had to get the hell out of there.
History records that the townspeople and farmers were generally astonished by the civil behavior of Lee's army which traded or paid fair prices for everything they got. There was one case of an army unit swarming a hat shop in McConnellsburg, taking hats and leaving beat up headgear in exchange. Their CO ordered the men to either return the hats they took or pay for them. They did.