I've listed four major causes. These are cumulative, not mutually exclusive.

You can't compete with aviation on price currently. Not while its environmental costs are graciously overlooked and left out of the ticket price. This is a political problem.

Comfort, reliability, sustainability, elegance, ease-of-use: those are the major selling points for night rail travel done right. This includes legroom, luggage space, and the ability to move around outside of your seat and to toss and turn in bed¹ — something current generation sleeper trains provide. If your goal is to cram as many people as possible in every metre of train length, you are optimising the wrong parameter at the wrong time.

1: The coffin beds in these hotel pods are clever from a space filling perspective, but I fear I would be nastily banging my knees several times in the course of the night, and that's ignoring the fact that sleeping that low close to the rails is just not pleasant.