Canandian waters are equivalent to Canadian soil. Such a baby would be a dual US-Canadian citizen at birth, but SCOTUS has not yet clarified whether or not the child would be NBC eligible to be POTUS, per the scholars I trust. Scholars I trust say such a citizen, born outside the US, would be a “statutory citizen” not a constitutionally defined NBC and thus not POTUS eligible.
Kreep and Taitz argued the NBC issue before Judge Carter and he said he was offended that a child of a US serviceman (John McCain) on active duty on foreign soil, as in your example, would be a citizen but might not be NBC eligible to be POTUS. Current State Dept. regs pretty much say the matter has not been resolved by SCOTUS.
The John Jay letter to George Washington rationale for the NBC clause makes avoidance of “foreigners” as CIC (with dual citizenship and dual loyalty) the primary reason for the clause.
You trust your scholars and the courts will trust theirs. They probably won't be the same scholars.