Until you take a baby to the vaccine clinic, the nurse googles which vaccine to give at his age, and blindly trusts the highlighted AI snippet at the top.

Not a fictitious example.

I saw a doctor use AI to do some maths recently. I checked it and it was right, but trusting LLMs to do statistics is not a good idea.

Sure, but that's an incredible level of incompetence, that I can only see would be expose otherwise outside of AI use. The entire list of what vaccines to give to what age group could fit on a single piece of paper.