You mean like what this guy had to say?
"I cannot undertake to lay my finger on that article of the Constitution which granted a right to Congress of expending, on objects of benevolence, the money of their constituents." -- James Madison
Was that before or after he supported Jefferson on the massive national roads project? Is that an "object of benevolence?" I admit, I can't find much about JM supporting government funded almshouses---but then again, I haven't done a whole lot of primary source research into JM. But he did go from being a Federalist to being a "Republican" then back to being, in terms of his policies, a Federalist, so it wouldn't surprise me if he had voted for Virginia laws supporting public almshouses. Indeed, at the time, HIS OWN politicies toward "disestablishing" churches (the primary source of charities0 argued that it had to be the state's job.
In another sentence it says Congress has the power "To regulate commerce with foreign nations, and among the several states, and with the Indian tribes;"
Now, I will agree that a strict reading of these does not contain the words "offer humanitarian assistance," but neither is it prohibited, and under "providing" for the "general welfare of the United States," it can be argued that limiting the spread of disease and buidling international alliances through aid easily falls under either category. For example, "providing for the common defense" does not strictly allow us to sell or give weapons to another country; but common sense tells us that Washington or Adams would have no trouble arming Indian tribes who would fight the British---or other Indians, and indeed, that is exactly what we did under MADISON and MONROE.