There are dozens of sources which refer to the white elephant. Do a simple google and pick a couple.
https://www.google.com/search?q=how+was+buddha+conceived
Here is one source - http://www.kbi.com.au/cgi-bin/engine.pl?Page=page.html&Rec=38
The Buddha was conceived into the womb of his mother, Mayadevi (by taking the form of a White Elephant descending from Tushita and entering the womb immaculately).
“The Buddha was conceived into the womb of his mother, Mayadevi (by taking the form of a White Elephant descending from Tushita and entering the womb immaculately).”
Is it possible that they were referring to the size of his soul? Not the physical manifestations.
BTW, that "family" was chased out of India about 200 AD and went back home.
When we know a historic person and someone says he was conceived by an elephant, it is necessary to reject that belief as untrue ~ and totally irrelevant ~ just crazy talk by ignorant people on a level with the stupid rulings by mullahs out in the desert.
An example ~ I have a book about the use of a mildly hallucinogenic mushroom (amanita muscaria) where the writer deduces that instead of Jesus being a real person the Apostles were into using this mushroom. He even cites scripture to support his argument.
Obviously that belief is not part of Christianity.