The transporter is fiction. Without knowing how it works, nothing can be said about it in reality.
But what's the difference between "you" and your youthful self? The state of your body? Memories? All those things are things "you" experienced. None of them were ever the "you" that experienced them. You're conflating your "self," your experience of "I," with your personality. But they aren't the same - "you" are aware of your personality. In fact, "you" are what is aware your personality has changed over time.
I'm thinking that this is a presumption. Perhaps "you" and "I" are ephemera, and the continuty of self is a consequence of the physical persistence of our bodies, and in particular our brains. Well, it's a great privilege, while it lasts.
As to this, I might reply more particularly. Of course now all I have is memories, not just of what I experienced, but what I THOUGHT. And in particular my thoughts of my future self. I even seem to remember certain moments in certain places where these thoughts occurred. But, it is so remote from me now! Frozen in time! I guess time is the ultimate mystery.