Life doesn't have to be rare to answer the paradox. For life to visit other planets it must be intelligent.
Intelligent life is obviously exceedingly rare. One has to go no further than our own planet to observe this fact. If you consider life on earth from the bacteria through humans, the odds against intelligent life developing, even on a hospitable planet like ours, is in excess of a billion to one.
The way I look at it, life has been on the Earth for about 4 billion (4x 109) years, intelligent life about (say) 100,000 (105), and radio-using life about 100 (102).
So the odds against intelligence would seem to be 40,000 (4x104) to 1, and against radio-using intelligence about 40,000,000 to 1. Not quite a billion, but...