There are planets and moons essentially made of water. So while a comet could be a source, more likely a comet is made of similar stuff to the rest of the space debris.
The universe seems to be made of water and hydrocarbons. Basic building blocks of the universe.
Building colonizable worlds will probably exploit that fact — start with a debris pile, piling up star system debris into an Earthlike orbit and with an axial spin, followed by piles of water ice, then finishing it off with piles of frozen nitrogren and oxygen. As the temperature rises then stabilizes, add in Earth organisms in an appropriate order.
Colonies would be constructed on big flat shield-shaped chunks of mineral foams (extruded in space from more debris) and dropped in. They’d ride the very large tides.