Yes but you defeat your own argument by saying it is a nuisance before it becomes a nuisance. It shows that the reason isn’t really the reason.
There is precedent elsewhere. This is not the first mosque on the planet after all.