How does HDCP work over DisplayPort? I guess HDCP is a different spec from HDMI itself?

Yes, HDCP is seperate from HDMI and DP.

The source and the sink need a HDCP-licence. Both devices have embbed keys that get exchanged to estabish a encrypted channel. Without the licence you can't get the required key material.

AFAIK, you can even sell HDMI devices without HDCP. Practically though, every entertainment device needs HDCP support.