Countries that were not under an empire also had famines and other disasters (China had a civil war with 50 million killed, IIRC) as well as endemic practices like slavery.

Sure. But we're discussing Victorians here.