Prediction: Non-determinism will become acceptable in areas we used to expect accuracy.

For example we will accept 'probabilistic bookkeeping' because it's cheaper than requiring ledgers to balance to the penny.

But this leeway won't be equally applied. Powerful institutions like banks will use “probabilistic models” to decide they probably don’t owe you that refund, but if they decide you owe them money, they will still hold you to every cent.

Nondeterminism for the powerful, determinism for everyone else. Yay!

Agree with your prediction in many cases but not bookkeeping. The whole point of bookkeeping is to balance the ledger to the penny so people would toss bookkeeping altogether before accepting a 'probabilistic' output. Agents will be used to accelerate the recon process though, or maybe they will become advanced enough to provide a (correct) deterministic result.

A bunch of people are going to become "acceptable risk" and "cost of doing business". Companies will opt to get it 95% right for cheap over getting it 100% right, and for 95% of the population, it will be good enough.

Companies can decide today that they don't need near-perfect accuracy in bookkeeping to save a few bucks and no one does that. One of the major factors is regulatory requirements. Even without I'm sure investors would apply a hefty discount to any public company that decided to save a few pennies on accountants while sacrificing accuracy.

Don't forget that this will likely be paired with rubber-stamping one-sided arguments maskerading as quality control processes where some maliciously-biased oversampling of (probably paid-for) good reviews takes place in order to consistently reach the conclusion that there's nothing wrong going on.

The bookkeeping can often be fully automated, if you care to do so, so there probably isn't a point.

People have used machine learning for fraud detection for a long time at this point. They do tolerate the false positives.

From what I remember reading on some tutorial about Random Tree classifiers banks on the USA have to justify the specific reasons why a credit was denied, so hence why blackbox models cannot be used for this.