> Blacklisted the hardware a KVM guest never has or doesn't need

I'm a little surprised that matters; shouldn't Linux be lazy loading drivers only when it sees hardware?

Most of them can be compiled into kernel OR as a module based on your config, some configurations require things to be built in. A lot of the platform stuff is usually compiled in to make booting faster and initramfs easier