It may well be that their was nothing "wrong" with your brother at all, as you say.
Whatever roots psychology may have had in science has long since been left behind. It has become a religion.
In some ways your statement is valid. But I can tell you that from over 35 years of exposure to my brother and cousin that they do have a condition of sorts. Eulalia, compulsive handwashing, and facial tics were childhood traits that meant they were teased unmercifully.
My cousin cannot even hold a job; his parents years ago put him in an apartment. He can live alone but he receives disability payments to pay for food and utilities -- and the model airplanes he builds. Plus he must be protected from predatory types as he cannot understand such motivations and is easily manipulated.