Except for their psychological impact, and (like all other weapons) their inhumanity, there is nothing particularly devastating about them.
As you stated, the psychological impact would be greatest on the civilian population. Jews being gassed again? Don't want to even think about the ramifications of that.