You should dress nice all the time. It just makes you feel good when you've dress yourself well.
The nicest (ie, most comfortable) clothes I own are work boots, jeans, and flannels. I feel absolutely great when I wear them. And I'm a lot more productive.