you can also have removed layers of models and keep the same score in benchmarks [1].

i feel that sometimes a lot of the layers might just be redundant and are not fully needed once a model is trained.

[1] https://snats.xyz/pages/articles/pruningg.html