This has little to do with Romney being a Mormon and a lot to do with him being perceived as a liberal Republican from the bluest of blue states. Four years ago, the GOP-E shoved RINO McLame down are throats. We held our collective noses and voted for him, and when he lost, many of us said "never again" because our conservative principles mean more than whoring out our vote out for the inside-the-beltway GOP-E.
Four years later, the GOP-E not only demanded again that we support a RINO who was all but a democrat when he served as govenor of Massachussetts, but they dissed the Tea Party and conservatives in the process. I suspect that many conservatives decided to stay home rather than compromise their principles again, and that cost Romney the election.