My call is to find a gun range which rents weapons and has the models you are interested in. Nothing is really comparable to taking the test drive. Finding out how a weapon fits your hand while shooting is really the most important data point and this cannot be related by someone else. Also trigger pull is crucial. One of the most telling signs of a cheap weapon is a bad trigger pull. While this can often be fixed by a trigger job from a good gunsmith, this does depend on model.
This weekend I finally shot a Glock again. I had shot one quite a few years ago and remember it not suiting me because it did not fit my hands well. Having a bit more extensive opportunity to shoot one this weekend after even more years of shooting experience, I’ve noticed this was still the case. The grip and configuration just doesn’t feel good. I was still reasonably accurate, but I would not own one of those guns.
Personally I’m a big fan of SIG guns. However I don’t have any of their polymer framed models so lacking in first hand experience I can’t directly comment on the model you are considering. I do know well and quite like their metal frame pistols (P220, P226, P229). I’d assume they are up to the SIG quality level I am familiar with, but then you know what they say about assume.
Oh, and I would also counsel the .40 S&W over the 9mm as you consider calibers. I’m a partisan of the larger hole school of self defense thinking. Personally I favor .45 ACP in a carry gun, but that doesn’t appear to be in the realm of what you are considering.
Second everything said by drbuzzard.
What you buy comes down to what feels comfortable to your grip when in use. Your physical size should not be an issue for .40 cals which my stable is mostly full of and the P226 is my favorite. Not only for trigger, but for endurance in sand, dirt and muck. The P226 is kinda big to lug around, but I am a bigger guy so it hides well under loose clothes or 511 gear.
Like the .45s and have a handful of those in 1911 frames, the Kimber Ultra Pro Carry II really carries nice.
Good luck