Are you familiar with the work of, say, Bandura or Vygotsky?
You will find that in all cultures, the lower class children do not learn as quickly in school and do not appear as smart. Part of this may be due to inherent differences in intelligence, but part is also due to their environment in the home - they are exposed to less vocabulary, less conversation, less reading, and on the whole, less cognitive stimulation.
There are educational methods that work, but many good teachers don't want to work in inner city conditions, and frequently those districts don't have the money needed to implement extra instruction.
"but part is also due to their environment in the home - they are exposed to less vocabulary, less conversation, less reading, and on the whole, less cognitive stimulation."
I believe you are correct.
A human being deprived of all stimulation during childhood would no doubt test as a very low IQ person, at least as we define it and measure it.
Recent insight into how we learn things (neuron formation as a direct result of stimulation) suggests that lack of stimuli during early formation of the brain may well intelectually cripple the individual for the rest of his life.
The upside of this research is that it has been demonstrated that continuing stimulation can indeed preserve and sometimes increase our cognitive abilities as we age, in the absence of organic dysfunction.