Actually the conjectures of Copernicus and Galileo regarding heliocentrism were not based on mathematics, and provided little mathematical advantage over the Ptolemaic system.
They stuck because they were good inferences and led to good research. But they were the result of imaginative thinking, not logical necessity.
This is pretty common in science.
Galileo's work tended to be more precise than that of Aristotle because he used mathematics. Mathematics may be employed in connection with a philosophy and the two together may work well as science. There is no need to maintain mathematics as a strictly separate discipline in order to accurately perceive and explain the world around us. It takes a peculiar philosophy to engage mathematics in total separation from other factors attending to human reason.