Student Handcuffed After School's AI System Mistakes a Bag of Chips for a Gun (theguardian.com)
- Reference: 0179868214
- News link: https://news.slashdot.org/story/25/10/25/0733242/student-handcuffed-after-schools-ai-system-mistakes-a-bag-of-chips-for-a-gun
- Source link: https://www.theguardian.com/us-news/2025/oct/24/baltimore-student-ai-gun-detection-system-doritos
> Taki Allen was sitting with friends on Monday night outside Kenwood high school in Baltimore and eating a snack when police officers with guns approached him. "At first, I didn't know where they were going until they started walking toward me with guns, talking about, 'Get on the ground,' and I was like, 'What?'" Allen told the WBAL-TV 11 News television station.
>
> Allen said they made him get on his knees, handcuffed and searched him — finding nothing. They then showed him a copy of the picture that had triggered the alert. "I was just holding a Doritos bag — it was two hands and one finger out, and they said it looked like a gun," Allen said.
[1] https://www.theguardian.com/us-news/2025/oct/24/baltimore-student-ai-gun-detection-system-doritos
Cops are dumb (Score:2)
AI is dumber. Though if they were the Flamin' Hot Doritos the mistake may be understandable.
They have to be (Score:3)
The job is boring as shit. If you score too high on an intelligence test they won't hire you because you're likely to quit after training.
We don't pay well enough to counter that so there you go. Cops are dumb by design
Re: (Score:2)
Wtf are you on about? If "they score too high on an intelligence test" they get to go on to a bright career as a detective and/or rise through the command ranks. Police forces always need bright bulbs. The starting pay kinda sucks and the job is dangerous which is why they usually don't get the best & brightest.
Re: They have to be (Score:2)
You clearly aren't American...
Re: (Score:2)
There was a single specific police department right next to the FBI HQ that decided they didn't want anybody over 135(?) IQ because the FBI would sniper them the moment they got a year's experience.
Re: (Score:2)
> The starting pay kinda sucks and the job is dangerous which is why they usually don't get the best & brightest.
It's is a dangerous job, often thankless, and subject to the whims of local and national politicians. They really need to be paid well from the start. Some places ( [1]Seattle for one [publicola.com]) are starting to recognize that... probably because it's getting harder to recruit.
[1] https://publicola.com/2025/10/21/new-police-contract-will-boost-starting-salaries-to-almost-120000/
Re: They have to be (Score:3)
The state cops in my state make up about 33 percent of the top 1000 people on the payroll. They make more than presidents of the state colleges. But not the state university. They make more than the vast majority of professors either at college or university level.
It is not uncommon to have a decent sized group of state cops making 300k or more.
âoeWe work wicked hard and do a lot of overtime.â
Sure you do.
Re: They have to be (Score:1)
The job is only dangerous in the big cities. I donâ(TM)t know what percentage of police are in big cities but I bet you most police in the majority of towns in America have never shot their gun. As far as the pay is concerned if you look at the budgets of most towns I bet you with overtime and other perks the highest paid employees are usually in the police department.
Re: (Score:2)
I think the problem is more that cops have to do all sorts of evil things as part of enforcing capital and whatever bigotry is codified where you are, which means anyone who joins is either too stupid to notice, or too evil to care,
Re: (Score:2)
Sure, but that's beyond dumb. Any moron knows the difference between a hand with a finger extended holding a bag or Doritos and a gun.
As punishment, they should be required to report to the front of the cafeteria and be paddled by the student in front of the entire student body.
The local news can make a day of it.
Human on the loop required (Score:4, Insightful)
You must - MUST - have a human double checking this. Dispatching any kind of response without human review invites catastrophe.
Re:Human on the loop required (Score:4, Interesting)
If you have humans double-checking your AI, then you don't save money. Never gonna happen.
Re: (Score:2)
Do you then suppose that double-checking an potential false positive is somehow more expensive than sending a squad to a school?
Re: (Score:2)
Sending the squad to a school is already paid for. It doesn't hit the budget. Assigning review duties to AI alerts? Gotta pay somebody for that shit.
Re:Human on the loop required (Score:5, Insightful)
> Gotta pay somebody for that shit.
Police are paid for by taxes, and public school is paid for by taxes. Everyone's already paid. It is pure criminal negligence that allows machine vision to automatically call police. The student's parents should sue the school, and the school administrators should be prosecuted for filing a false police report.
Re: (Score:2)
> Police are paid for by taxes, and public school is paid for by taxes. Everyone's already paid.
How is it the least bit relevant how they are paid? The fact is you will have to pay one ADDITIONAL person to double check AI alerts if it happens often enough. Its not that dispatchers are dumb or that police are dumb. Its that its not their job to second guess AI. That is a manager problem. They got AI and then let it send a squad car to deal with an armed student without anyone in the chain of command having the responsibility/authority to countermand it.
That problem is systemic. Police departments are
Re: (Score:2)
Or just have the leader of the squad that's about to go threaten a kid over a bag of Doritos give the photo a look first. It pays for itself by leaving the squad free to pursue something more relevant and worthwhile. The 30 seconds it would take to look at the picture isn't even enough time to get to their cars.
Re: (Score:2)
> by leaving the squad free to pursue something more relevant and worthwhile. The 30 seconds it would take to look at the picture isn't even enough time to get to their cars.
None of which has any reality in the context of police operations. The squad sent was likely the one already closest to the kid. Have you noticed how fast cars are going with their sirens on? You expect them to carefully look at a photo and decide whether that is really a gun? Moreover, the alternative would likely being cruising doing nothing relevant or worthwhile unless they happened to catch a speeder. Or, more likely, Walmart security would have had to wait an extra 5 minutes for a different squad to p
Re: Human on the loop required (Score:1)
Wait until the AI will start dispatching killer drones.
Re: (Score:2)
well it may take an case that makes to the court room where the source code is put under question.
Re: (Score:2)
AI source code isn't the issue. The models are the issue, and they're just a bunch of math.
Re: (Score:2)
That's what was supposed to happen, but the school's administration was not present to review the footage and cancel the alarm, hence the police came in force.
Re: Human on the loop required (Score:2)
Cops might need a human double doing the checking but most people could do it themselvesâ¦
Re: (Score:2)
> You must - MUST - have a human double checking this. Dispatching any kind of response without human review invites catastrophe.
According to TFS, the cops on the scene had a copy of the photo, so this should already be the case.
> They then showed him a copy of the picture that had triggered the alert. "I was just holding a Doritos bag — it was two hands and one finger out, and they said it looked like a gun," Allen said.
Hopefully, they had already surmised this was a false alarm, but guessing they had to cover their asses and check it out and follow through as though it was a real situation, since it was flagged by the system. Not defending this, but can see why it happened. Had the video simply been originally viewed by a person, who would have realized it was a finger, it would have ended there. On the downside, peopl
Re: (Score:2)
Enter the vibe checkers.
Re: (Score:2)
That's fine. Except that if it turns out its really a gun, when they're going over all the missteps involved, the time spent waiting for somebody to go over the result manually will absolutely come up.
Just like traffic cams (Score:5, Funny)
Its "almost" like you need an officer to review the footage before issuing the ticket so we arnt humiliating people and wasting anybody's time with nonsense.
Re: (Score:2)
Reviewing footage costs a lot of money. There's no room in the budget for it.
Re: (Score:2)
They looked at the picture afterward...just not before bothering to go on site.
If they had looked at it afterward BEFORE bothering to go on site, they could have saved everyone a lot of effort.
Re: (Score:3)
The AI already told them what they saw. When you remove the cognitive load of recognizing what something is and then you prime people with a set answer, they'll believe it much if not most of the time. If I told you that the lyrics to "Kiss the Sky" were "Kiss This Guy" and you'd never heard of the song before, you'd hear "Kiss This Guy" when you listened. I told my kids that "Smack My Bitch Up" was "Smack My Pitch Up" and they believed it for years.
Re:Just like traffic cams (Score:4, Informative)
Anecdote: I received a red-light camera ticket in the mail a few years back. Pre-AI, they were using some sort of automated license plate reading software, which confused an 'O' with a 'Q'. They dutifully attached a copy of the picture of a late model GMC pickup truck with my vehicle plate and description. A 45 year old Toyota Landcruiser. Not even close.
My mistake was in calling the municipal court and having the error fixed clerically. Which saved me time but ended up not having me challenge the ticket in court. Which would have made the error a public record and a data point useful for others to challenge the systems accuracy.
Where is the picture? (Score:4, Interesting)
Where is the picture? I want to see what AI thought was a gun, and police, who should KNOW the difference in appearance between a bag of Dorito's and a gun, felt it was enough to go on to draw guns and arrest the student. Then I want to know what brand of AI cameras were in use.
Re: (Score:2)
If it could possibly be interpreted as a gun and the AI recognition primes, the viewers, they will usually see it as a gun.
Re: (Score:2)
The police were acting on the assumption that "what they heard" was correct, which they always do, and always did, before AI.
"What they heard" was some kind of school administrator placing an emergency call about a student with a gun.
That administrator hasn't said anything publicly yet that I've seen, but the AI company is claiming their own human staff had reviewed the images, flagged it as a false positive, and communicated that false positive status to the school admins.
The question is who knew what and
new rule no outside food but you can buy from vend (Score:1)
new rule no outside food but you can buy from vending with X2 mark up
Re: (Score:2)
With chip bags carefully designed to not look like guns.
Re: new rule no outside food but you can buy from (Score:2)
Fingers are kind of a wildcard though
Re: (Score:2)
Yeah, after the redesign the bags will look like knives.
Re: (Score:2)
Doritos is a subsidiary of Pepsi-Cola. The school probably had an exclusive marketing contract with Coca-Cola.
Good to know (Score:2)
Kids can now sneak guns into school inside Doritos bags.
Police: We're not falling for that one again!
Re: Good to know (Score:2)
The boy who cried Doritos
Re: Wearing camo from head to toe... (Score:2)
Good idea, blame the victim. Next up, they'll be drone striking motor boats off the coast of Venezuela, because drug traffickers might also use boats.
Not an accident (Score:2)
These displays of extreme authority are not an accident, they're a feature of a system that intends to remind you of the presence of constant surveillance.
Summary is bad, it was even stupider than that (Score:5, Insightful)
From [1]https://www.wbaltv.com/article... [wbaltv.com]
> "I am writing to provide information on an incident that occurred last night on school property. At approximately 7 p.m., school administration received an alert that an individual on school grounds may have been in possession of a weapon. The Department of School Safety and Security quickly reviewed and canceled the initial alert after confirming there was no weapon. I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local precinct for additional support. Police officers responded to the school, searched the individual and quickly confirmed that they were not in possession of any weapons.
So the school resource officer (SRO) called the cops after the school had already determined it was a false positive and cancelled the alert . So the cops showing up, and this kid having to get handcuffed and searched, is all thanks to the SRO - who really deserves to be fired and quite possibly sued.
[1] https://www.wbaltv.com/article/student-handcuffed-ai-system-mistook-bag-chips-weapon/69114601
That's not AI failure! (Score:5, Insightful)
That's really stupid human oversight.
Re: (Score:3)
That's how humans use AI.
Re:That's not AI failure! (Score:5, Insightful)
Actually, it's both.
Re:That's not AI failure! (Score:5, Insightful)
In other words, the AI failed, and the humans also failed.
Re:That's not AI failure! (Score:4, Informative)
No. If you want to avoid false positives, you have to accept false negatives, and conversely. Set the recognition to be "super cautions" and it's going to make mistakes that say "Maybe a gun there". This is literally inevitable.
What's really stupid is that the police looked at the picture of a Doritos bag and a couple of fingers and didn't realize it was a false positive. (Or more likely didn't even bother to look at the evidence before flying off the handle.)
Re: (Score:1)
> What's really stupid is that the police looked at the picture of a Doritos bag and a couple of fingers and didn't realize it was a false positive. (Or more likely didn't even bother to look at the evidence before flying off the handle.)
From TFS, it doesn't sound like the police saw a picture until after the fact. The way it reads, the AI flagged the frame in the video and called the police. The police were responding to a call for an armed person. If you're responding to a call for a possible shooter, it's unlikely you will be grilling the caller, on site, before trying to stop a potential mass shooter.
It seems to me that the company who makes the AI should require a person to review the video once the call goes out. That way they can