Not contradictory at all. These days private browsing for most people just means (1) don't save the browsing history and (2) log me out of all websites temporarily.

But as the other post notes, it goes further (than, for example, Chrome Incognito) in ways that can break sites. Incognito means exactly what you said, while Safari Private Browsing means more.