That's the key point here. You won't be able to sue a judge who does that, but he could still be impeached. And probably would be.

You would probably have a really good shot at an appeal, too.

"Probably" would be impeached? The bar for impeachment and recall would appear to be exceptionally high, often rising to the level of "national news coverage and extended outrage" (witness the judge on the Brock case).