The pilot is the very last word on who may and may not fly with him. If he says they go, they go and as far as I know there is no recource except perhaps from the airline to the pilot.
Thank you 101voodoo for confirming that.