Usually that means the record was mastered differently (because you literally physically can't make a record as "loud" as a CD).

It's not the CD's fault, it's the mastering engineers.

Yeah it depends on where the producer expects the CD to be played.

99% of music is made to be played on radio / in car etc., a noisy environment, where you don't want to be adjusting the volume knob all the time. So the dynamics are stripped in mastering phase.

Music that gets pressed on vinyls isn't mastered for car-play, but home stereo equipment, so it makes more sense to have larger dynamic range.

CDs have objectively lower noise floor (less hissing), and more dynamic range (difference between loudest and quietest note), but it's the mastering that usually destroys the sound. And nothing can be done about it on consumer end. Except find a less remastered version of the album in a thrift store that isn't scratched to oblivion.

There's really no reliable way to tell if a CD is going to have high dynamic range, except perhaps niche audiophile studios like https://www.stockfisch-records.de/sf12_start_e.html, but https://dr.loudness-war.info/ has fantastic list of records with their dynamic ranges, so you can check before you buy, and you can also explore and find new stuff to use to listen to your speakers ;)