// WIRED US/UK — INTELLIGENZA ARTIFICIALE
This Is Flock’s AI Search Tool for Cops
Flock Safety’s latest search tools give police several ways to find people on camera, not just the license plates on their cars. The capabilities sit inside a suite of software now at the center of dozens of police-stalking cases. WIRED collected and analyzed that software, and created a mockup of what officers see when interacting with the system.
Among the tools is an AI-powered watchlist. An officer can cordon off an area of a city, and make the cameras inside it run a continuous, automated search for anyone fitting a written description.
Over the past year, Flock has faced a backlash to its technology. Police departments across the country have canceled contracts, and city councils that once approved Flock have quietly voted the cameras back out. Two members of Congress wrote to CEO Garrett Langley after a Texas deputy searched more than 83,000 cameras for a woman who had an abortion. And Illinois found the company had let federal immigration agents reach state camera data in violation of state law.
Flock answered in August with a package of changes that shortens the default period over which data are retained, requires case codes for every search, and adds automated auditing to catch officers misusing the tool—all of which Flock says will be mandatory by the end of the year. WIRED examined the code those promises run on, but much of what governs a search happens on Flock’s servers. Crucially, the analysis found that Flock’s guardrails may discourage some abusive uses of its tools, and will record them, but will not stop them.
Experts who study automated screening and police technology say Flock has given its AI a job no technology does reliably. Flock’s system decides, out of sight of the department using it, which descriptions of a person an officer may search for—and no screen operating at that scale is accurate, the experts say. Further, nobody outside the company can measure how often it is wrong, or in which direction. This puts users in a bind: The company itself warns officers that results may be inaccurate or incomplete and should not be relied on alone, but puts the “risk of any inaccuracies” on the officer running the search.
Some of those decisions the experts find harder to defend than others. Flock’s system scores an officer’s written description against several categories of sensitive content, including race and religion, and “political, social, and cultural expression” is the only one that will not stop a search.
Tom Bowman, policy counsel with the Center for Democracy and Technology's security surveillance project, says the one sensitive category Flock guards with a speed bump instead of a stop is the one courts have guarded most closely. Political and cultural expression is “one of the most protective areas” under the First Amendment, he says, and it “seems like that actually has the least concern from a policing perspective.”
WIRED sent Flock a detailed description of each of its findings along with 16 questions. The company responded with a written statement saying its search tools are built to help police find relevant information “while maintaining clear safeguards around how the technology can be used.” Queries are evaluated against its content policies, it says. The statement does not describe how its AI model was built, what instructions it works from, or what separates one verdict from another.
Data flows in both directions. When the software flags a search, it sends Flock the language the officer used, the category and confidence score the model assigned it, and whatever the officer did next, whether continuing, dismissing, or disputing the warning. Flock’s statement does not address how long those records are kept, who inside the company can see them, and whether they are used to evaluate or improve the moderation system.
Flock’s interface holds several tools, and each one works differently. A search that uses text to find images, for example, produces a statistical ranking of which foo