A pardon can, of course be refused, but if the pardon doesn’t spring the pardonee it is because the powers that be have decided that he doesn’t have to leave prison if he doesn’t want to or if the pardon is post punishment the refusal has no effect at all.
Right. Most pardons are requested, so oddball cases very rarely come up.