It not warm as in ”warmer than the typical living space”, but it is warmer than zero Kelvin, so heat can be extracted from it.

Doing that takes energy, that’s why it is called a heat pump. That moves heat from the water to an already warmer place, against a heat gradient, just as a water pump moves water against a gravity gradient.

If the water were warmer than your typical living space, they wouldn’t need a heat pump; a water pump to pump the water closer to where heat is needed would be sufficient.

Practically, the water would need to be somewhat warmer than 0℃ because you don't want it to freeze and clog the plumbing after you have extracted a useful amount of thermal energy. :)