Of course, but that doesn’t affect the morality of the thing. If someone told you “I have no self-control when it comes to drugs”, what are you going to say to them? “Oh, well ok, have fun then.”
No, you say, “I hear ya man, but ya gotta try your best, ‘cause that stuff will mess you up.”
I agree, ultimately only God can truly judge how significant a control you have over your own impulses. It seems like the significance of a sin depends on how strong or weak an individual is. People can become stronger of will over time but their struggle should count for something when coming to judgment.
Just as adultery would be a greater sin for me than for say a sex addict. I have much better control and should be able to resist with little effort. The addict on the other hand is a starving man at a banquet.