Not sure why you said they are Arabs--there are almost no Arabs in Kosovo--used to be a few missionaries, but they're mostly gone due to lack of success. And those are the guys you can be sure were NOT there.
Religion in the Balkans--even among the Serbs--is to a large degree more an expression of heritage and culture than it is a practiced faith. It is also normal among the Albanians to attend each other's (Sunni, Orthodox, Bektashi, and Catholic) major religious events just as it is normal for them to hardly ever attend the normal Friday & Sunday services for any of those religions.
But of course. This must be how you can account for all the dynamited Serb Orthodox churches in Kosovo. The Muslims were simply practicing their faith of Mohammed the dynamiter.