Why are you having lunch with the Pope when there are hungry people out in the slums?
Good question.
I would say, Jesus had private meals with his disciples regularly. It fed them nutritionally and spiritually to go out and live the Gospel. This meal with the Pope should also be a time to gather with the Lord’s blessing to obtain sustenance and fortification for the journey.
It is a good question, and your answer is probably about what he was looking for. Not “You shouldn’t be here,” but “What do you think you should do next?”