Yeah that seems silly. I was a quiet-room sleeper until I met my wife, who needs some kind of white noise to sleep. I eventually adapted and now sleep much better with noise than without (at least subjectively), but that change took a while, at least several months. I found it quite difficult to sleep with for a while.

I slept without noise for years until I needed it for a while for (reason) and now it feels difficult to sleep without noise. Acclimation’s a real effect