I don’t know what other states do, but when I got my ballot at my voter registration address, there were 2 envelopes inside the mailed envelop. The larger envelope also had my correct registration address and a place to put my signature and the date. The smaller envelope had nothing that identified me, and I was to put my filled out ballot into this and seal it, put it into the signed envelope which did not need me to apply a stamp, and then mail it before the deadline date.
Thus, the key to fair counting was for poll watchers from both parties observe the outer envelopes being opened and the blank envelopes removed and all put together. Then watch all the blank envelopes be opened and the ballots inside be sorted and counted. So the key is the final opening and counting process. Each party has to do a good job of having the right people there at the end. One mailed ballot complaint I heard was that when the outer envelope was opened the plain white envelope had already been sealed so they could not vote by mail. Perhaps instructions should be included saying that if the plain envelope cannot be used, one should substitute some other plain envelope to contain the ballot for submission.
I live in NV and for our mail in, June primary, the outside envelope had my party affiliation printed for all to see. All they have to do is throw it away. No one would be the wiser.