Is diffing really that expensive even in complex applications?

You have the diffing, but you also just have the various random bits of churn from building up the thing to diff etc.

Say what you will about "plain" COM models they don't cause a bunch of random memory allocations when you drag-n-drop an element in a list down two spots