Except Patton. And of course Roosevelt, Truman and certain general officers knew about the Manhattan project.
Patton was one General out of hundreds. There was no other General that thought it was a good idea at the time, at least on the American side. The German side certainly wanted to do it. However, who is to say we would have had any better chance to succeed where the Germans and before them Napoleon failed?
Yes, the President knew about the Manhattan project. They also knew they only had three bombs (one was used for the test) and there was no guarantee that they would have surrendered after dropping the second bomb. McArthur, Nimitz and the rest were planning an invasion of Japan. It’s a good thing the Japanese didn’t know we only had two bombs. It would have taken a long time, perhaps 6 months or more to build more bombs and the Army and Navy weren’t going to wait around. Besides, Russia was already invading Manchuria and kicking the Japanese out of there. In six months time who knows where they would have been. Perhaps they land on the Japanese soil and they had no qualms about sacrificing soldiers.