Then again, your neighbors might have come here to get away from the kind of Muslim orthodoxy found in Muslim counties... they might want to fit in, but now are afraid. You called it in - then left them alone. That's the right reaction. You did the best thing possible - noticed something odd - and called it in.
People who want to fit in would use ordinary house numbers and make themselves as inconspicuous as possible, not do things which stand out in the neighborhood...
My take is that this is the sort of neighborhood take-over commonly asserted by 'protected' groups. Move in, make the neighbors uncomfortable so they'll move out, move in more people of similar nature, lather, rinse, repeat.
Only the hyper-tolerant (people whose political idealogies demand that they discard all semblance of rational judgement to 'prove' they are 'tolerant') and the oblivious would not be affected.