If the problem is that weld heating has altered the steel properties, drilling out the weld wouldn’t seem to do much.
If a “splint” could be applied that were of the same order of strength as the original steel that extended beyond the heat damage due to the weld, it seems it to me it might help. But, a washer and nut wouldn’t seem to fit the bill!