Isn’t it surmised that the Magi gifts supported the holy family in Egypt? Not exactly refugee status.
One can be a refugee and still have financial resources.
It IS fair to call Jesus, Mary, and Joseph refugees because they went into exile in Egypt to escape persecution.
It IS reasonable to suppose they used the gifts of the Magi as financial support, and did not become beggars or recipients of some sort of Egyptian benefit program.
It IS necessary to add that they went back to Nazareth as soon as the risk from King Harod was over. They did not remain any longer in Egypt than they had to.
Yes, the sacrifice for a baby boy in Israel at the time was a one year old lamb (from Leviticus 12:6) but if the parents were too poor, they could sacrifice 2 doves. Mary and Joseph sacrificed 2 doves so they were poor, and the gold and other gifts certainly helped them when they moved to Egypt.