I have done "illegal" work on my home as an owner/contractor without permits, sometimes using day laborer help (not illegal immigrants, though), and ALL the work I did was beyond code.
The work I did would have cost double had I gone through proper channels. I'm not saying I am wrong or right in what I've done-- but it just comes down to what I can afford, and the knowledge that I can do it as good or better than a licensed contractor.
Then I'll say it - you are right. It's your property. Screw the government parasites.