“...warming it up for the simple reason that I don’t like fogged up windows and freezing my butt off in winter. “
That’s an excellent point I hadn’t even thought of. On freezing winter mornings, if the conditions are right you will get yourself killed trying to drive off before you get heat from the engine because the windshield will ice over inside. Those of you who live up here in snow country know what I am talking about!
Make sure you aren’t closing the outside vents and you shouldn’t have any frost. Even when it’s below zero, I don’t have any issue with fogging up as long as the heater fan isn’t set to recirculate the inside air.
That said, do what works for you. You’re paying for your own gas.