There's an overlooked aspect of AI performance:

Due to the very diffuse statistical nature of the mapping of the input to the output, some will be experiencing extraordinary appropriate useful results while others will be experiencing crap. And due to the subjective nature of quality and quirky topographies of the internal terrains of these systems and the haphazard control protocols, the spread of resultant value (so to speak) of these appliances may not be understandable as a normal distribution. Scenarios we might judge as lying close to each other on the input terrain may generate very peculiar bumpy results of output terrain.

One of the most damning considerations of this technology is that gross uncertainty about result quality may be manipulated by the appliance maker leading to wild patterns of exploitation without any prospect of accountability.

As there's no pretense of formal function, there can be no verification, and so there's no methodology for determining quality: a presumably working appliance may appear reliable until it isn't with no way to understand the inflection. Those with insider control and information can game customers with no chance of accountability, but the system can't be understood.

The problem with magic beans is guessing what they'll grow into.

Meanwhile, the seismic movement of business towards these private and completely mysteries silos of "knowledge" (which is a grotesque misnomer) may arrive at dead ends from which they can't retreat.

This is a far greater danger than a malicious bot because a great migration into the dead end is not just well under way, but regarded as a brilliant prospect which we must run into at all deliberate speed.

+1 for "...may not be understandable as a normal distribution." "Rogue" might just be the operators wishful thinking. Hammer nail (something from a prior life), politics isn't normal: http://athena.m3047.net/elections/dist-not-normal/distributi... Time to complete software projects is another. :-/

So that people know what we mean. There's also Wikipedia: https://en.wikipedia.org/wiki/Normal_distribution