Rain usually just cakes the dirt. Unless it's a downpour, it doesn't move with enough velocity or volume to remove the dirt. So the dirt just gets damp, maybe moves a few cm with gravity, then dries as a mud cake.