Respectfully, you're a generation off. Two of Mitt Romney's great-great grandfathers (Miles Park Romney and Helaman Pratt) practiced polygamy, but his grandfathers weren't polygamists.
It's interesting that Helaman Pratt claims to have become a polygamist because LDS prophet Brigham Young told him to:
“On April 20, 1874, Helaman, on the advice of Brigham Young, married Dora Johanna Dorothy Wilcken [Anna Johanna Doratha Wilcken] as his second wife.”
Mary Pratt Parrish, “Helaman Pratt,” in Hatch and Hardy, Stalwarts South of the Border, 543-52. Cited by award-winning LDS historian Todd Compton in his online work on Romney's polygamous heritage.
Romney's family left Deseret - the theocracy Brigham Young attempted to establish in the Utah Territory (which led to the Utah Wars with the U.S. Army), and went to Mexico to help establish a LDS polygamist colony. Romney's grandfather, Gaskill, brought the Romney family back to the U.S. in 1912 due to the violence of the Mexico Revolution.
You are right, I had accepted it after seeing it on a few threads and had not verified it myself.
Great grandfather Miles Park Romney had fives wives and 31 children?