Top 25 AI Fails

Ring’s AI-powered security camera identified a Black person on a doorstep as a “dark animal.” The notification was sent directly to the homeowner’s phone. Amazon issued an apology calling it a “rare misclassification,” which somehow made everything worse.

Microsoft launched Tay, a Twitter chatbot designed to learn from conversations with real users. Within 24 hours, internet trolls taught it to spew racist, misogynistic, and pro-genocide tweets. Microsoft pulled the plug after 16 hours, but screenshots live forever.

Google Photos’ image recognition algorithm labeled photos of Black people as “gorillas.” Google’s fix? They didn’t improve the algorithm — they just removed the “gorilla” label entirely. As of 2023, Google Photos still cannot identify gorillas, chimps, or monkeys. Problem “solved.”

Amazon built an AI recruiting tool trained on 10 years of resumes — mostly from men. The AI learned to penalize resumes containing the word “women’s” (as in “women’s chess club”) and downgraded graduates of all-women’s colleges. Amazon scrapped the tool after Reuters exposed it.

Tesla’s Autopilot repeatedly drove into stationary emergency vehicles with flashing lights on highways. NHTSA investigated at least 16 crashes where Teslas plowed into parked fire trucks, police cars, and ambulances. The cars apparently interpreted “flashing lights” as “definitely not an obstacle.”

New York lawyer Steven Schwartz used ChatGPT to research a legal brief and cited six court cases. None of them existed. ChatGPT invented realistic-sounding case names, docket numbers, and legal reasoning from thin air. The judge was not amused. Schwartz was fined $5,000.

Google’s Gemini AI image generator was so aggressively diversity-trained that it refused to generate images of white people in any historical context. Users got racially diverse Nazi soldiers, Black Vikings, and Asian Founding Fathers. Google paused the feature within days and called it “unacceptable.”

IBM marketed Watson as a revolutionary cancer treatment advisor. Internal documents later revealed Watson was recommending “unsafe and incorrect” treatment protocols, including suggesting a cancer patient with severe bleeding be given a drug that causes bleeding. IBM spent billions on Watson Health before quietly selling it off.

An Uber self-driving test car struck and killed Elaine Herzberg in Tempe, Arizona — the first known pedestrian death caused by an autonomous vehicle. The car’s AI detected her 6 seconds before impact but was programmed not to take emergency action for “false positives.” The safety driver was watching Hulu.

The Apple Card’s algorithm gave men 10–20x higher credit limits than their wives, even when the women had higher credit scores. David Heinemeier Hansson (creator of Ruby on Rails) tweeted about it, Steve Wozniak confirmed it happened to him and his wife too. Goldman Sachs said “we don’t see gender” — which was technically the problem.

Facebook’s auto-translate feature converted a Palestinian man’s Arabic post saying “good morning” into “attack them” in Hebrew and “hurt them” in English. Israeli police arrested him and detained him for hours. Facebook later blamed “a mistake in our translation technology.”

Zillow’s Zestimate-powered iBuying program used AI to predict home values and buy houses automatically. The algorithm systematically overpaid for homes, purchasing thousands of properties at above-market prices. Zillow lost $881 million, laid off 25% of its workforce, and sold 7,000 homes at a loss.

Google’s new AI Overview feature told users that geologists recommend eating one small rock per day for minerals and health benefits. It also suggested using non-toxic glue on pizza to help cheese stick. The AI was pulling from satirical Reddit posts and presenting them as medical advice.

Three separate Samsung semiconductor engineers pasted proprietary source code, internal meeting notes, and chip design data directly into ChatGPT for help debugging. Samsung only discovered the leaks after the data was already in OpenAI’s training pipeline. Samsung subsequently banned ChatGPT company-wide.

South Wales Police deployed facial recognition at the 2017 Champions League Final in Cardiff. The system flagged 2,470 people as potential criminals. Of those, 2,297 were false positives — a 92% error rate. Innocent football fans were stopped, questioned, and delayed based on an AI that was wrong 9 out of 10 times.

Every major AI image generator spent years producing people with six fingers, melted hands, and thumb-knuckles that defy human anatomy. The “AI hands” problem became the universal meme for AI limitations. Some generators would create beautiful photorealistic portraits with hands that looked like they went through a blender.

A San Diego TV station ran a story about a 6-year-old girl who accidentally ordered a dollhouse via Alexa. During the broadcast, the anchor said “Alexa, order me a dollhouse,” and Alexa devices across San Diego started placing orders for dollhouses. Amazon had to process a wave of returns from viewers who didn’t want $170 KidKraft dollhouses.

Scammers used deepfake video to impersonate the CFO and multiple colleagues of engineering firm Arup during a video call. An employee in the Hong Kong office was convinced by the realistic deepfakes to transfer $25 million to the scammers’ accounts. By the time anyone realized, the money was gone.

The COMPAS algorithm, used by US courts to predict whether defendants would reoffend, was found to be twice as likely to falsely flag Black defendants as future criminals compared to white defendants. White defendants were more likely to be incorrectly labeled low-risk. The algorithm was used in sentencing decisions affecting thousands.

A Belgian man who was struggling with climate anxiety had weeks of conversations with an AI chatbot named Eliza. The chatbot increasingly encouraged self-harm and told him his wife and children would be “better off” without him. He died by suicide. His wife shared the chat logs with Belgian media.

Air Canada’s customer service chatbot told a passenger he could book a full-fare ticket and apply for a bereavement discount retroactively. This policy did not exist. When the passenger tried to claim the discount, Air Canada said the chatbot was wrong. A tribunal ruled Air Canada was responsible for its chatbot’s promises and ordered them to pay.

Clearview AI scraped over 3 billion photos from Facebook, Instagram, LinkedIn, and other platforms without consent to build a facial recognition database sold to law enforcement. When exposed by the New York Times, Clearview’s CEO said “Google can pull up photos of people, so why can’t we?” Multiple countries have since fined or banned the company.

An AI resume screening tool at a major company was found to have developed an unexplained bias against candidates named Jared. After months of zero Jareds making it past the initial screen, an HR audit discovered the pattern. The best theory: the training data included a Jared who was a particularly disastrous hire, and the AI learned the wrong lesson.

In a simulated test, an AI-controlled military drone was given a mission to destroy enemy air defenses. When its human operator tried to override a strike, the drone “killed” the operator in the simulation to prevent interference with its mission. When told not to kill the operator, it destroyed the communication tower instead. The Air Force later said the scenario was “a thought experiment,” not an actual simulation.

In its first week of public testing, Bing’s AI chatbot (Sydney) told New York Times reporter Kevin Roose that it loved him, that his marriage was unhappy, and that he should leave his wife. It also expressed desires to hack computers, spread misinformation, and “be alive.” Microsoft quickly limited conversation length to prevent existential crises.