You know why that is? Most people think you worship a mortal - Mary. Everything is Mary, not Christ. There's Mary's beads, statues, prayers, ornaments of Mary, but very little has to do with the one who bought you.
I always thought the Catholics made Mary their Goddess. That's all I ever saw. If one listens closely, they'll notice Mary is used to get to Christ in the end.
*shrug*
If "most people" are too ignorant to educate themselves on the Catholic faith, then I suppose they can think whatever they like.
That's because those people are too ignorant and lazy to find out about others' beliefs. They are arrogantly blissful in their ignorance.
Ever hear of the Eucharist?
Some people think Protestants worship the Bible, too. Everything is "Bible this" and "Bible that", but nothing about how Christianity was practiced prior to the existence of the Bible...
That's why Catholics follow scripture AND tradition. If the Holy Spirit was truly present at Pentecost, then the oral tradition that fueled the early Church must have been of Divine origin as well...