When we got married, I worked, she stayed home with the kids, and our standard of living was determined by what I made. She learned to be a good cook and we didn't eat out that much. We deferred buying a house until my career had advanced to the point of affording it, which happened to be after the birth of our second child.
The use of the word "afford" defines the speakers priority scale. Saying "I can't afford X" really means "I have other things that are more important to me than X, and those things consume all my available time and money".
Makes sense.