Traditionally they have been. Several data centres in London are colocated in office buildings for example. I used to manage servers in one that most tenants wouldn't even have known about.

AI focused data centres also could, but require a higher amount of power and cooling per rack, so it's less practical. The point is not that they don't use more than a typical office, but that they don't use the world-ending amounts people assume they use.

The operators should still be made to pay for the externalities, and regulated to ensure they don't make life miserable for people around them.

But that doesn't mean we need to stop building them.