| ▲ | Animats 10 hours ago |
| That's not a Flock error. That's a law enforcement error. Flock isn't responsible for a bogus warrant in the system. The woman probably has a good case for damages from the police department for doing it twice. That's sloppy police work. |
|
| ▲ | aftbit 10 hours ago | parent | next [-] |
| You should read Normal Accidents. "Human error" is an excuse. It isn't really actionable. Instead of trying to fix the "human error", you must design the system such that it does not permit life-threatening errors. Sometimes that means a tradeoff in terms of efficiency, cost, or something else. Often this tradeoff is not one that people in charge of the system are willing to make, unless they are forced to do so by a larger institution or force. To really drive this home: 1. What steps did Flock Safety take to ensure this warrant was valid and current, and that the plate number was correctly entered? 2. What steps did the police officers take to validate the warrant against the APB hit from the camera? 3. What process is in place to ensure incorrect warrants are not entered or mistakenly kept live for longer than intended? My guess is that the answer to all three of these is basically "none". |
| |
| ▲ | jkestner 9 hours ago | parent | next [-] | | A human must always be held responsible, but a tool must expose its limitations to enable human judgement. I put this in the same basket as Autopilot, which already fails with its name. I’ve seen too many cases where people intuitively trust the magic technology instead of verifying it. The tool shouldn’t not present itself as magic or AI or whatever the current term is. | |
| ▲ | andrewla 9 hours ago | parent | prev [-] | | 1. I for one do not want "Flock Safety" in the loop for this at all. Police departments can flag a license plate in the system; that's on them. Flock is just a tool for departments to communicate what would previously have been an APB and have better detection than relying on chance sightings. 2. The vehicle was correctly identified by Flock as being the one flagged, and verified by the local PD that it was the one that was flagged. That said, they could have verified that the flag was still meaningful before converging on the vehicle. 3. This is a real problem and this is really the core of the oversight. When the lawsuit over this is filed, it seems like 80% of the liability should attach to #3, and 20% to #2, and essentially none to #1. It's hard for me to see why Flock is even relevant to this discussion except to drum up outrage. | | |
| ▲ | aftbit 9 hours ago | parent [-] | | "But for" Flock Safety, this would never have happened. Similar to saying that Tesla is responsible for designing a safe AutoPilot even though there's still a driver in the loop who is responsible. | | |
| ▲ | andrewla 9 hours ago | parent [-] | | No, this is like saying that we should hold the fax machine liable for one PD faxing an APB to another one, or the concept of radio waves responsible when a PD dispatcher announcement for a a BOLO. | | |
| ▲ | aftbit 2 hours ago | parent | next [-] | | IMO it really isn't that similar. There's a fundamental difference between a fax machine and Flock Safety. One is a device that you can pick up off the shelf at an office store and which is used by all manner of customers for all manner of things, while the other is a bespoke system designed for and sold to meet public safety use cases. Furthermore, it's a new evolution of technology. Sometimes quantity has a quality all its own. It was one thing when tracing a call or tapping a line required a technician to run around the electromechanical switch with a volt meter or a human to listen to your calls 24/7. It's an entirely different thing when it can be done automatically and cheaply at scale. Before, you could hope to hide in the noise, and unless you were doing something "really bad" (or you had powerful enemies), you probably wouldn't be targeted. We're on the precipice of being technically able to have cheap, total enforcement of most laws. Should we actually do that though? Will the societal changes be broadly good or bad? That's a question we should be asking, not just jumping in with both feet and pretending that the technology is neutral in the outcome. For example - the introduction of radios in police cars had a huge impact on public safety and played a huge part in bringing an end to the era of the public enemies. No longer did you just need to outrun the posse chasing you and get out of town for a few weeks until the heat was off. Now you needed to slip out of the dragnet being coordinated over radio around you. That seems like it was probably broadly a good thing ... but even then, it required cost and human risk to accomplish. As we move further into the era of AI policing, we'll leave all of that behind. | |
| ▲ | chaps 8 hours ago | parent | prev [-] | | Are fax machines designed within a tight feedback loop specifically between law enforcement? (Actually yeah. Chicago Police uses an on-prem 3rd party blackbox fax service to avoid those records being FOIA'able. But that's beside the point.) |
|
|
|
|
|
| ▲ | lkey 9 hours ago | parent | prev | next [-] |
| American police have no obligation to protect, serve, or tell the truth. The police steal more money than all petty thieves every year. Qualified immunity makes accountability a distant mirage. The police operate exactly as they are allowed and encouraged, by the system they labor within, to behave. Your words 'sloppy' and 'error' imply that a 'careful, correct' approach fixes these ills. History and experience shows us that this hypothetical good will not be produced until laws and incentives more broadly, change. |
|
| ▲ | cdrnsf 10 hours ago | parent | prev | next [-] |
| How often are police actually held accountable for anything? |
| |
| ▲ | fluidcruft 10 hours ago | parent | next [-] | | Likely never again with this Supreme Court. | |
| ▲ | NewJazz 10 hours ago | parent | prev [-] | | The municipalities that employ them are held accountable (and often prevented by unions from firing them). |
|
|
| ▲ | netcan 10 hours ago | parent | prev | next [-] |
| IDK. This can be a banal point. Product, technology, implementation, police department, policing method, workflow. These aren't discreet. Incidents involve all of them. Responsibility is on people.Technologies cannot be "found guilty." But... this isnt' a trial. There are consequences to doing policing any particular way, and that's the subject. Doing it this way... using baby skynet for command and control has advantages, like response time. It has disadvantages, like false positives. "Law enforcement error" is technically correct, but not informative about the matter at hand. The error |
|
| ▲ | ertgbnm 10 hours ago | parent | prev | next [-] |
| It's a failure on all fronts. I think it's an indictment of Flock in the same way that full body scans are a scam. If you are always scanning for illnesses/crimes you will inevitably find tons and tons of false positives without ever meaningfully improving outcomes. The juice isn't worth the squeeze. Flock has been around long enough that if it had a meaningful benefit to safety, they'd be able to prove it in numbers. Instead we just have a long list of instances where it has made the world worse. |
|
| ▲ | duxup 6 hours ago | parent | prev | next [-] |
| Her car was flagged as being involved with a homicide. What should the cops have done? Ignored it? I'd argue Flock making that incorrect association is similar to a false accusation and the consequences should be on Flock. |
|
| ▲ | 9 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | eclipticplane 10 hours ago | parent | prev | next [-] |
| This behavior would not be possible at scale without Flock's technology. There's enough blame to go around to hit Flock and the PD both. |
|
| ▲ | oceansky 10 hours ago | parent | prev | next [-] |
| Plenty of blame to distribute. |
|
| ▲ | sophacles 10 hours ago | parent | prev | next [-] |
| Flock amplifies existing bugs. Sounds like Flock is the error. |
|
| ▲ | shimman 10 hours ago | parent | prev | next [-] |
| No this is bad logic. Flock is absolutely responsible and they should not be releasing or selling products that are this poorly made. Saying that the responsibility is only on the user is extremely myopic. It's also dismissing the role Flock has in how they make and design their software. If they can't handle this maybe they should be disbanded as a company or nationalized for better public control. |
| |
| ▲ | lukan 10 hours ago | parent | next [-] | | "Flock is absolutely responsible and they should not be releasing or selling products that are this poorly made." I am not a fan of flock, nor police surveillance state, but what could they have done better here? "Milwaukee police later admitted the alert should have been cleared out of the system days earlier, but an employee simply forgot to remove it. " The system seemed to work as intended - that numberplate was flagged for investigation with homicide - and gave the alarm as intented, nothing more. The problem was wrong data given to them by humans. | | |
| ▲ | nogridbag 10 hours ago | parent | next [-] | | Yeah I'm confused. The title of the article is wrong. There was no "Flock Camera Glitch" according to the article. The human in the loop did not properly delete an invalid license plate number from one of the databases the flock system checks against. The issue issues are with: 1. Human in the loop 2. Third party database police use for homicide info 3. Police response I don't see what the license plate reading technology has to do with any of this. It seems like the only thing working reliably. | |
| ▲ | gopher_space 9 hours ago | parent | prev | next [-] | | > I am not a fan of flock, nor police surveillance state, but what could they have done better here? We'd need to take a look at what they have access to and how they're ingesting that info. I'd start with caring greatly about false positives and not automating department onboarding. Working with various counties I've seen situations where someone needs to tell you a specific table dump exists to get an accurate count because that's how they published deletions back in 1983. There are no standards and the people who created these systems are all dead. > The problem was wrong data given to them by humans. This is always a problem. It's never not a problem. The question is how Flock chooses to address the problem. | |
| ▲ | JoshTriplett 10 hours ago | parent | prev | next [-] | | > I am not a fan of flock, nor police surveillance state, but what could they have done better here? Flock should stop existing. I hope people help them with that, enthusiastically. | | |
| ▲ | genewitch 10 hours ago | parent | next [-] | | don't worry, Axon wants to put all 90,000 CCTV cameras into a database for dragnets, too. And i'm sure there will be startups, possibly even mentioned here, that want the same thing. There's a lot of money in cyberstalking and real stalking. see: Facebook's valuation | |
| ▲ | nickthegreek 9 hours ago | parent | prev [-] | | Legislation is how that happens. There is no other way. And these stories are what help get that ball rolling in the direction we want. | | |
| ▲ | esseph 9 hours ago | parent [-] | | > Legislation is how that happens. There is no other way. There are plenty of ways if you are creative enough. Somebody could start putting Flock executive license plates in with erroneous police data. |
|
| |
| ▲ | esseph 9 hours ago | parent | prev | next [-] | | Which would not have been acted on if those charges were ever in the flock system to begin with. Which means they wouldn't have flagged the car and pulled it over. A couple of days go by and the error is out of the system. | |
| ▲ | shimman 8 hours ago | parent | prev [-] | | They can design a system that doesn't enable these behaviors. Good grief these excuses are pathetic. This is a purposely made system with specific outcomes, these are the outcomes. Saying there is nothing we can do is pathetic behavior and just shows that maybe you (nor Flock) should be in charge of these things if these are the outcomes they find acceptable. |
| |
| ▲ | fluidcruft 10 hours ago | parent | prev | next [-] | | The flock cameras were clearly misused. Misuse is 100% user error. | | |
| ▲ | shimman 9 hours ago | parent [-] | | Then you should design systems to ensure this type of user error doesn't happen. I'm sorry but this is software and all these excuses are straight up pathetic. If you don't know how to design a system to prevent this, DON'T MAKE IT! Good grief our industry is absolutely pathetic with these excuses. | | |
| ▲ | fluidcruft 2 hours ago | parent [-] | | Did you actually read the article? It's like you are demanding a car to not drive when you push the gas pedal. | | |
| ▲ | shimman an hour ago | parent [-] | | No it's more like demanding crumple zones, seat belts, and third brake lights. Just because you have zero standards or ethics does not make this acceptable at all. This was a purely created system that was purposely designed exactly for these outcomes. You do not have to create such a system. There can be more accountability. Have standards please. This is impacting real human lives here because you refuse to have standards. |
|
|
| |
| ▲ | daedrdev 10 hours ago | parent | prev [-] | | Them is flock responsible for reducing auto thefts and should be counted too? |
|
|
| ▲ | andrewla 10 hours ago | parent | prev [-] |
| If a human had spotted the license plate that had been flagged with an All Points Bulletin or an Amber Alert the same outcome would have occurred. It's hard to see why Flock is even mentioned in this except for clickbait purposes. From the article, it appears that the vehicle was not misidentified; it was correctly identified but incorrectly flagged. |