You can't possibly know that this interpretation is true by hearing only one side of the story. The number of ways that a so-accused manager has to explain this as being completely appropriate, thus making the complaint look unfounded are myriad. It's the kind of thing that just gets you permanently ignore. Can't take it, then find another job.
Um, that's one of the reasons for going to HR - so the manager has an opportunity to tell his side of the story.