Do not take him back. That would be a death sentence for him.
I already decided not to take him back. His last littermate died shortly after we took him to the vet.