Another serious weakness was thick windshield framing that reduced forward visibility, a problem during combat.
I would think it was because they could not reliably produce thick curved bubble windshields without a high defect rate or failure rate in desired mass quantities.
One of the problems that you have with an aircraft that flies that fast is getting rid of the heat developed by the friction of the air passing over it. It’s a materials problem fundamentally. Sometimes all you can do is mitigate the problem thru some fancy engineering.
Contemporary aircraft the the US Navy’s F8U Crusader were limited by the heating effects on their wind screens. The Crusader III, which never reached squadron service, was capable of dash speeds in the Mach 3 region. The test aircraft were showing problems with windscreens beginning to fail.
The A-12/SR-71 was entirely designed to shed heat. The engines were reportedly capable of pushing that air frame much faster than the Mach 3.35 top speed, but the pilots dare not push too hard for too long. You can do a little more with a strategic recon aircraft that flies with less frequency than a carrier fighter.