The first time was when we lived in Newport Beach, CA. When I was in my early teens, my cousin Ron was a cabinetmaker at a local boatyard, and was working on the Duke's boat (a converted minesweeper). I went to visit him for lunch one day. While I was there the Duke showed up to look things over, I remember he was very friendly, and even made an attempt to laugh at my juvenile attempts at humor (I do a pretty good John Wayne impersonation even today).
The last time was when I had landed a part as an extra and happened to run into him a couple of times during filming. We talked a couple of times and he told me that I reminded him of some skinny kid who did a really bad imitation of him a few years prior. I asked him if it was at the boatyard, and he said, "Yes! Dammit! It WAS you. You son of a bitch!" We had a laugh about it, and he told Denver Pyle (also on the set) about it. They both had a laugh on my account.
If I’m not mistaken that converted minesweeper of Wayne’s was moored from time to time just off a small island (Bay Island) in Newport-Balboa harbor. My parents vacationed there during the summer most every year.
Awesome story!