It doesn't even need malicious intent. If nobody rational is monitoring it, all it will take is a bad datapoint or hallucination for your door to get kicked in by mistake.

Plus there is inherent biases in datasets. Folks who have interactions with Medicaid will be more vulnerable by definition.

To quote the standard observability conference line "what gets measured gets managed".