"Are you also still without understanding? Do you not see that whatever goes into the mouth passes into the stomach and is expelled? But what comes out of the mouth proceeds from the heart, and this defiles a person.... These are what defile a person. But to eat with unwashed hands does not defile anyone." (Matt. 15:16-18,20)
Yes, the context was the washing of hands. But he's obviously not talking about hand-washing per se; he's talking about "defiled food" (which passes into the stomach and is expelled).
From a health perspective, eating "unclean foods" may well have been ill-advised -- there was a good chance of getting trichinosis from pork, and red tide could often poison shellfish, for example; but it is difficult to square such food safety issues with what Jesus actually said.
Jesus is pretty clear in saying that a person is not made unclean by what he eats; even pork or shellfish wouldn't defile him in God's eyes. It's what is already in a person's heart that defiles him.
One can also make a pretty good case that Peter's vision in Acts 10 is confirmation of this view; not to mention Paul's discussions of eating food sacrificed to idols.
The context is indeed ritual washing. And Jesus told his disciples and us what he meant:
Mat 15:16 And Jesus said, Are ye also yet without understanding?
Mat 15:17 Do not ye yet understand, that whatsoever entereth in at the mouth goeth into the belly, and is cast out into the draught?
Mat 15:18 But those things which proceed out of the mouth come forth from the heart; and they defile the man.
Mat 15:19 For out of the heart proceed evil thoughts, murders, adulteries, fornications, thefts, false witness, blasphemies:
Mat 15:20 These are the things which defile a man: but to eat with unwashen hands defileth not a man.
The issue was ritual washing of hands, a tradition that Jews invented, not something scripturally commanded. The food laws, on the other hand, were something that Christ himself commanded.
Your interpretation, though common, is a mistake in that it is done through a prism of tradition that disagrees with biblical precepts and practices.