Oh, yeah. My late father joined up in 1937 as a buck private in the 106th Illinois Cavalry. And yes, they rode horses, McClellan saddles and the whole bit - he cried (and he wasn't alone) when, in 1941, they took them away. But the Army had mules all throughout WWII, and in the very rough country in Italy, in particular, that was a primary means of supply.
I still have his original sidearm, a .45 ACP revolver. And the 106th still has an association. About 10 years ago Dad went to a reunion and came back shocked - "Damn, boy, those guys are old!" Uh, yeah. But they'll smile when they read this story.