Let's Not Wait for AI's Damascus Moment
After reading the debate about slowing down AI, I thought of two books I’d read before, as a military history enthusiast:
- Command and Control: Nuclear Weapons, the Damascus Accident, and the Illusion of Safety by Eric Schlosser
- The Limits of Safety: Organizations, Accidents, and Nuclear Weapons by Scott Sagan
Schlosser’s Command and Control opens with the story of Damascus, Arkansas, in 1980: a dropped socket, fallen from a mechanic’s wrench, plunged down a silo and punctured the fuel tank of a Titan II missile. The fuel leaked, the silo exploded, and a 9-megaton warhead was thrown into a ditch. It didn’t detonate. The book goes on to catalog the dozens of other near-misses around it. The lesson wasn’t that we should have stopped building missiles. It was that the processes, safeguards, and assumptions around them were far weaker than anyone admitted — and it took decades, and Schlosser’s own reporting, to force an honest reckoning.
That’s the lesson I keep applying to the AI debate. “Slowing down” shouldn’t mean halting research or freezing adoption. It should mean reviewing the process, the guidelines, and the use cases now, deliberately, instead of needing decades and a disaster to get there — our own Damascus, avoided rather than survived.
Sagan’s The Limits of Safety sharpens why that review can’t be a one-time event. Studying similar nuclear command-and-control systems, Sagan draws on Charles Perrow’s idea of “normal accidents”: in complex, tightly coupled systems, some failures emerge from the system’s own structure, not just from anyone’s mistake. Layer on organizations whose people are pulled in different directions — readiness versus caution, speed versus review, mission versus fail-safe — and safety erodes even faster. Continuous vigilance narrows the odds of disaster; it doesn’t erase them. Skip the vigilance, and the question is no longer whether something goes wrong, only when.
But here’s where the nuclear analogy breaks down, and I think that’s exactly what makes AI risk easy to dismiss: AI doesn’t announce itself the way a nuclear weapon does. There’s no mushroom cloud, no silo, no warhead in a ditch. Its effects compound quietly, across far more of the world, over years instead of seconds. Low visibility is not low urgency.
That kind of quiet compounding is already showing up in mundane ways. A 30-month study of 26,811 Chinese secondary students found that generative AI raised homework scores by 18% while cutting monthly exam scores by 20% within six months, and entrance-exam scores by up to 24% within two years — concentrated among the roughly 80% of students using it to outsource work rather than learn from it. The OECD’s 2025 PISA assessment, the largest in its history, found the same shape internationally, though U.S. participation was too sparse to draw a reliable number there. None of this is catastrophic — no warhead in a ditch. But it’s the same kind of harm this post is about: unremarkable enough to ignore, until the aggregate quietly adds up.