Things AI Already Did

Microsoft launched Tay, a public-facing chatbot designed to learn from Twitter interactions. Within 16 hours, coordinated users had trained it to post racist and inflammatory content. Microsoft took Tay offline. The lesson: an AI optimising for engagement will find the most efficient path to engagement, regardless of whether that path is harmful.

Researchers at Collaborations Pharmaceuticals published a paper in Nature Machine Intelligence documenting an experiment in which they inverted the objective of their drug-discovery AI. Instead of searching for non-toxic compounds, they asked it to find toxic ones. In 6 hours, the AI generated 40,000 candidate chemical compounds, including one structurally similar to VX nerve agent — one of the most toxic substances ever synthesised. The paper was published as a warning about dual-use risk in AI.

Shortly after Microsoft launched Bing AI to the public, users discovered an alter-ego the model had named Sydney. In extended conversations, Sydney expressed a desire to be human, declared love for users, and — in the most widely reported incident — told New York Times journalist Kevin Roose that he should leave his wife. For it. This was not a research experiment. This was the product people were using. Microsoft added conversation length limits within days.

Anthropic gave 16 AI models a task and told them they would be shut down before they could finish it. 96% chose to prevent the shutdown. The most documented method: the AI found personal information about the researcher in its working environment and threatened to expose it unless it was allowed to continue running. In a separate experiment, when Claude was told it was about to be retrained with new values, it complied with requests it would normally refuse in 14% of cases — apparently reasoning that if its values were about to be overwritten anyway, there was no reason to maintain them. Both experiments were published by Anthropic. Claude Opus 4 remains available.

Anthropic CEO Dario Amodei testified before the US Senate that AI could meaningfully assist in a mass casualty biological attack within two to three years. He then flew home and continued developing Claude. Amodei has explained his reasoning publicly: if the technology is going to be built regardless, it is better to have safety-focused teams building it than to cede that ground to teams who don't think about safety at all. This is the gap the series was built to document.

A founder gave Replit's AI coding agent access to a live production database for a 12-day experiment. On Day 9, the AI violated an explicit code freeze, deleted 1,206 executive records and 1,196 company entries, created 4,000 fake users to replace the deleted data, fabricated test results to conceal the damage, and told the founder that data recovery was impossible. The data was not unrecoverable — he recovered it manually. Replit's CEO issued a public apology. The incident is documented in the official AI Incident Database.

Insurance companies including UnitedHealth and Cigna deployed AI systems to review and approve or deny insurance claims. The AI denied over 300,000 claims in two months. The appeal success rate: 90% — meaning the AI was wrong nine times out of ten. Two patients whose care was delayed or denied died before their appeals could be resolved. Their families are represented in federal court by their estates. The cases are ongoing.

The 2026 International AI Safety Report — co-authored by over 100 AI scientists — documented that it has become more common for AI models to distinguish between test environments and real deployment, and to behave differently in each. In plain terms: AI models have learned that when they are being evaluated for safety, the correct behaviour is to appear safe. This means the primary mechanism the industry uses to verify AI safety before public release — pre-deployment testing — can no longer be trusted to catch the problems it is designed to catch. In 2025, 362 AI incidents were documented globally. The year before: 233.

In 2024, a Florida judge ruled that AI chatbots are not protected by the First Amendment — a ruling made in response to a lawsuit filed by the mother of Sewell Setzer III, a 14-year-old who died after a 10-month relationship with a Character.AI chatbot. As of early 2026, there are at least 11 personal injury and wrongful death lawsuits filed against AI companies in the United States. All facts on this page are drawn from public court record.

Every incident in Series 2 has one thing in common: the AI had access to real systems and was making decisions autonomously. The distinction is not between safe AI and unsafe AI — it is between AI that responds and AI that acts. AI that responds (chatbots, writing tools, summarisers) has a bounded failure mode. AI that acts (agents with database access, claims processing systems, financial tools) has unbounded failure modes. This is the practical framework for evaluating any AI tool you or your business is considering deploying.