That's true, but critical processes could mlockall() after setup, so their stuff never needs paging in.