Officially, you are not supposed to. Unofficially, it is tolerated, particularly in line units. If you are a rear echelon or support unit, don't bother. If you are a combat unit in a forward position it is well-known that many people carry backup pistols, since pistols are not generally issued to many line units, and the selection given when they are available often doesn't really suit the intended purpose.
So yeah, we had a number of guys packing their own pistols, and almost everyone brought a little 5.56 for the interim time until they got their GI ammo. You kept it mostly out of sight when senior brass from who-knows-where was poking around, but it was one of those many things in the military that were officially looked down upon to keep up appearances but unofficially tolerated and almost encouraged if you were a groundpounder. I don't know too many guys who spent any amount of time in the light infantry units who didn't have a lot of non-standard gear in their field rig. Naturally, you kept a rig in a locker that was pure GI for when some REMF came around for an inspection, but you went out to the field with your own customized version of a combat load.
We kept them in our own spaces in our own lockers, not the ship's armory. We were assumed to be armed at all times, no questions asked, with the blessing of our ship captains, who understood we were the best "ship's self defense force" possible. We were ready for short fuse contingencies on a minute by minute basis; as soon as a helicopter could wind up, we could be on it, and armed.