I think if someone followed me and my friend around and kept a transcript and summary of every conversation between and around us in a little notebook, I would have a basis to say my privacy was being violated, right?

Apple Watch doesn't maintain a transcript of every conversation. The user has to take action, and then it only goes back 15 seconds.

If someone is following you around within earshot and fiddling with... a notepad and pencil or a watch or a phone or a tape recorder, then yes, you have reason to be upset. I agree.

The ability of the Apple Watch to do what has always been possible doesn't materially change your privacy. [1]

Fundamentally, this is an accessibility feature, and Apple -- and other vendors -- have already shipped several similar ones. Were you equally upset by those too? AirProds already 'record' all audio around them! They can even be used as listening devices! https://support.apple.com/en-au/102479

This reminds me of parents getting hysterical when someone pulls out a DSLR at a kid's soccer game, while standing next to half a dozen parents who are all taking pictures with the phones and immediately uploading them to Meta.

[1] There are valid arguments to be made that this will be normalised and cloned... badly by Chinese and South Korean vendors without the privacy safeguards, at which point it is "too late" to put the genie back in the bottle.