If he was coming to preach infanticide, I assure you the university would not invite such a person. If he was invited to talk about secular issues that concern the audience, his private views would matter exactly nill.
At best it is a very grave scandal, giving harm to the souls of those who attend such an event (in that they may think such views are OK).
Is Obama coming to talk about the "beneifts" of abortion? If not, why is that a grave scandal any more than the pope praying in a syngaogue?
That Obama speaks is not a scandal at all, for reasons, more or less, that you articulated. Scandal is that he is honored as a lawyer, while being an avowed deathist.