I dont think we have restrictions here (SF Bay area) and a room in a shared apt/house is in many areas even more expensive ($1500-2000).

There’s pretty significant restrictions in most of the Bay Area. But even without restrictions, for decades the SF board of supervisors was actively demolishing SROs and trying to make housing more expensive so they had a better class of residents. https://ccsroc.net/s-r-o-hotels-in-san-francisco/

The most expensive real estate on the planet is in the bay area, and as with any dense urban area, it's all about what the market will support.

Forced open bidding would make one big difference in pricing. But those high salaries just make everything more expensive.