You are exactly right.
In the past I have personalized this argument with people. I try to use a named loved one of theirs. A daughter, granddaughter or someone close to them. I use the example of a pedophile wanting to have a sexual relationship them and then I ask the person what gives you the "moral authority" to say they can't have your relative?? Obviously our society has a "code" that finds it unacceptable - but the usual response is "because it's not right." If the person tries to list any reference to a bible they are trapped because bibles of old still have the listed sin of homosexuality which is "normal" to the brainwashed citizens.
When people refuse to have an established moral code they reap this "benefit" of "everyone does what is right in their own eyes." A pedophile, homosexual or bestiality - hey they are "sexual orientations" leave these poor people alone and let them "enjoy" their lives!