I believe you are 90% correct, but there are a few more factors involved.
Cell phones rarely get polled.
Caller ID helps to filter out unwanted calls and a lot of the poor get polled cannot afford it, and are more than willing to participate in the polls if they think it will get them more.
People with jobs are not at home during the day to answer the phone, and at night...they filter those calls from numbers they don't recognize.
In short, polling by phone has gotten to be the most inaccurate method available, yet it is the cheapest.
On the other hand you are onto something when it comes to members of groups who want their opinion to be more important than it really is ~ let's think about the GBLT, abortion rights and feminazi crowd. They want to get called by a pollster ~ and they answer every single time. Their party line talking points are firmly in their minds so they have no problem answering the questions. The result is that they have a 100% response rate while everybody else has a 9% response rate. That makes them 11 X more important than the others, per person polled!
Recently the gurus behind the GBLT gay marriage initiatives have figured this out. They say the polls say everybody in America wants gay marriage yet every time they run a referendum they lose! Some have suggested duplicitous pollsters are misleading the GBLT crowd so they'll spend all their money and go away.
But you're absolutely correct in surmising that polling by phone is just about the worst method anyone could use. And it's no longer cheap when you need to call 100,000 people to get 9,000 answers so you can set up a baseline on demographics vis a vis an opinion on some issue.