I’d be more willing to consent to these tools if I had confidence that the underlying software truly was built to honor my privacy.

Too many AI tools are built hastily for me to give my doctor’s (visibly awful) software the trust.

This is a challenge for me.

On the one hand, pre-LLM there was plenty of software out there for medical use, and they had to be HIPPA compliant, etc. I've worked with people who used to write that SW (they hated the job because HIPPA was so strict). The rational side of me is saying I shouldn't be biased against these, and that there's no way they would relax the requirements.

On the other hand, the emotional side of me screams: It's LLMs! Huge privacy concern!

I try to let the rational side win. I've always given consent. Especially because I liked my doctor pre-LLMs, and it seems silly to suddenly mistrust him and go to a doctor I don't know at all.

Indeed, I always turn this down as I don't consent. If you had a thorough read of the major providers' Terms of Service, you wouldn't either.

It's a shame, if they had an AI transcription tool that kept everything in-house I'd be much more comfortable with it. Or perhaps some kind of zero-knowledge cloud-based product where I hold the keys.