I don't know if I'm comfortable with any of this at all, but seems like having AI do "front line" judgments with a thinner appeals layer available powered by human judges would catch those edge cases pretty well.

This is basically how the administrative courts work now - an ALJ takes a first pass at your case, and then you can complain about it to a district court, who can review it without doing their own fact-finding. But the reason we can do this is that we trust ALJs (and all trial-level judges, as well as juries) to make good assessments on the credibility of evidence and testimony, a competency I don’t suspect folks are ready or willing to hand over to AI.

The courts already have algorithmic oracles for specific judgements, like sentencing:

https://en.wikipedia.org/wiki/COMPAS_(software)

I don't follow your reasoning at all. Without a specific input stating that you can't be your own victim, how would the AI catch this? In what cases does that specific input even make sense? Attempted suicide removes one's own autonomy in the eyes of the law in many ways in our world - would the aforementioned specific input negate appropriate decisions about said autonomy?

I don't see how an AI / LLM can cope with this correctly.

When discussing AI regulation, when I asked that they thought there should be a mechanism to appeal any determination made by an AI they had said that they had been advocating for that to go both ways, that people should be able to ask for an AI review of human made decisions and in the event of an inconsistency the issue is raised at a higher level.

To get to an appeal means you obviously already have a judgement against you - and as you can imagine in the cases like the one above that's enough to ruin your life completely and forever, even if you win on appeal.

Because historically appeal systems are well crafted and efficient? Please... at least read your comment out loud to yourself.