He lost most of his hearing was he was 20. You can see it in his body movements. Better almost to be born with a loss than lose it at that age.
Now that's an interesting comment, because I believe in "The Great Escape" there's a scene where he's pretending to be deaf, and a Nazi fires a pistol unexpectedly behind his head to test him, and he doesn't flinch (this could be a completely different movie and actor - Stalag 17? - but I think this is right). Anyway, if so, McQueen would have been using his lack of body movement to indicate his deafness, maybe as an inside joke.