When water freezes it expands, when the ice caps melt the water level goes down, not up.
Fill a glass with ice then water, cover with platic wrap and wait.....as the water melts the levels decreases.
If the ice is floating in the water, it displaces a volume of water equal to its weight. Because ice weighs less, it displaces less water volume than there is ice volume and the ice is exposed at the top. When it melts, the water level will remain the same. (Try it.)
That was not the situation during the Ice Age where precipitation in the form of snow recrystalized on the continents into ice sheets literally miles thick. From the Northern landmasses of Canada into the Dakotas (where I sit was once covered with ice well over a mile thick) and farther south than the Great Lakes, North America was covered with ice.
That did not displace any ocean water, but instead was removed from the oceans by evaporation and subsequent precipitation on land, where it formed those glacial ice sheets that planed the Canadian shield and carved the Great Lakes.
Greenland's continental glaciers and those overlying landmass in Antarctica are fine present day examples.
Did the pack ice extend further south? Almost assuredly, but still there was a net volumetric shift of water from ocean basins to land in quantities which simply could not have been sustained if the water was liquid (because it would have simply run off back into the oceans like it generally does today).
That volume of water tied up in ice on the continents is the amount of water removed from the ocean basins by being piled up, frozen, on the continents, and sea level dropped as a result.
The reason global warming alarmists are generally wrong about catastrophic sea level changes today with a small increase in temperature is that there is relatively little water tied up in continental ice sheets and glaciers (with the exception of Antarctica and Greenland), most is liquid or in a relatively small volume of alpine glaciers.