I got this insight from a previous thread : you can run linux with gui on the same specs as win 95 fine if your display resolution is 640x480. The framebuffer size is the issue

That and the fact that everything is 64 bit now. The Linux kernel is certainly much bigger though and probably has many more drivers loaded.

It is not one factor but the size of a single bitmap of the screen is certainly an issue.