The difficulty with Social Darwinism is that it can be taken at least three different ways:
You have both made interesting points about 'Social Darwinism,' which really has to be considered as an episode in the development of political philosophy with no standing at all in science.
My own view (for what it's worth) is that there is a fundamental category error in seeing an analogy between biological evolution (over a span of some billions of years) and the 'evolution' of human culture and institutions (over a span of a few thousand years). There are some cute 'analogies' between the two processes, but no real commonality between the underlying mechanisms, and I think arguing political/cultural/moral issues on old Darwin's back is to wander on to thin ice.
I like to think (oh alright, call it my 'faith' if you must) that the nature of our species is such that rational solutions ultimately prevail, and that core conservative ideas are superior, more efficacious, and do ultimately prevail because they are more rational. We had to oppose the dangers of Communism by military means, but its ultimate 'extinction' (see how easy it is to slip into Darwinian analogy!) is coming about because we have won the argument on rational grounds. The deleterious effects of big government, tax-and-spend programs, and all the rest of the liberal ideological craziness is defeated in the same way.
I am sorry that a minority group of self-styled Christians have such an issue with Darwin, believe he is the source of 'moral decay' or whatever--and also annoyed that these folks just refuse to engage with the science and do not acknowledge that a majority of Christians have no issue here at all. But I guess that's what these threads are for
Science, religion and philosophy have all been used this way.
A rational person would deduce that bad people cloak their deeds in the most respectable cloating available at the time.