While what you say is very true, asking the Holy Spirit to enter us if we are unclean or have impure motives will create much turmoil in ourselves.
Of course, and I've also written about that extensively. The Bible is filled with plenty of examples of people who mishandled God's power and paid a terrible price for it (Nadab and Abihu, and the sons of Sceva being two stark examples).