Lawyers are getting in trouble because they use AI and submit fabricated citations about fabricated cases as precedent. A bunch of charges were recently thrown out in Wisconsin because of this, and it's not the first time such behavior has made the news.

https://www.wpr.org/news/judge-sanctions-kenosha-county-da-a...

AI is indeed being understood to be an expert that replaces human judgement, and people are being hurt because of it.