Many Catholic Clergy seem to disagree with you on that point, at least to a certain point. Monsignor Cristiani talks about sortilege, and other demonic manifestions in his book (he mentions different things in addition to sortilege, such as infestations, and familiars, in addition to outright attack and possession). He certainly said in his tome that sortilege is possible, and described an incident which he said had occurred in the previous century, according to local Church records.
However, if I recall correctly, the sortileges he spoke of were afflictions that were cast upon a hapless victim, as opposed to the victim being compelled to do something against their will, and without their consent. That may be the difference.
As I said above, I am by no means an expert like Cristiani, Father Amorth, or Father Euteneuer. However, many Catholic clergy do seem to believe that there is indeed "something" to the idea of sortilege (even if there is no "official" church teaching upon the subject - something that I admit that I do not know either way). This may be one reason why we are not to have anything to do with paganism or witchcraft - we will just invite more of the same into our lives.
At the end of the day, I don't want to have anything to do with ANY of it. I keep my eyes on God, try to do His will, go to Confession and Communion, Serve Low Mass, and try to stay on the right path. That is MORE than enough to keep me occupied right there!!