They got your name with the credit card. With the zip code they can narrow it down and likely get your home address.
I use cash. Giving them my zip is no big deal for me.