Why are differential probes so expensive and why don't more scopes just have differential port built in?

For about $300 you can buy a Tiepie differential usb scope: https://www.tiepie.com/en/usb-oscilloscope/handyprobe-hp3

You can create a pseudo-differential input by combining two input channels on almost every scope. That's not the problem the differential probe is solving, though. The differential probe exists to provide a differential measurement between two voltages that may be isolated or significantly different than the ground voltage of your oscilloscope.

The ground lead on your probes is connected straight to the ground on the power cable. This gets new users in trouble when they're probing power circuits and they don't realize that connecting the ground part of the probe to something will cause a short to ground. If that ground clip pops off and brushes against the high voltage you're trying to probe, you get sparks and maybe a destroyed scope.

The differential probe provides isolation and rejects the common-mode (shared) voltage between the two probe points before it gets to the oscilloscope.

I don't know about that USB probe, but I prefer not to have single-purpose instruments that require their own desktop software to use.