If I were writing the creative brief for the AI apocalypse, I’d have to admit the campaign is pretty good. Start with a technology everybody has heard about but relatively few people understand well enough to evaluate. Make the consequences enormous, preferably existential. Recruit credible experts to explain why waiting is dangerous, give reporters a few memorable phrases like “rogue agents,” “catastrophic risk” and “loss of control,” then repeat them until they start sounding less like predictions and more like established facts.
Then offer the solution.
More control.
This part should be familiar to anybody who has spent time in marketing. You don’t begin by selling the product. You establish the problem the product solves. And if the product you’re selling happens to be government authority over artificial intelligence, “ChatGPT occasionally makes things up” isn’t much of a problem statement. Extinction, on the other hand, will get you through the first meeting.
Purely as a marketing exercise, the campaign brief writes itself.
- Problem. Artificial intelligence is advancing beyond our ability to control it.
- Stakes. Jobs, national security, democracy and possibly human survival.
- Villain. Rogue models, autonomous agents and unrestricted AI.
- Urgency. We need rules before the next generation arrives.
- Solution. Audits, mandatory safety requirements, restrictions on dangerous models and, yes, an actual government kill switch.
The only thing missing was the documentary. Netflix took care of that this week.
Apparently the apocalypse has a release schedule
On September 15, Netflix began streaming The AI Doc: Or How I Became an Apocaloptimist, a documentary that follows filmmaker Daniel Roher through the question of what artificial intelligence might mean for his new child and everybody else who would prefer humanity to remain an ongoing concern. Netflix describes the film as an investigation of how AI might affect his child’s life and “humanity’s future.”
Now, before somebody sends me an email explaining that I’ve uncovered the Netflix-Congress Industrial Complex, I haven’t. I haven’t seen evidence that Netflix coordinated anything with Congress, and a documentary roughly three years in the making obviously wasn’t commissioned last Thursday because somebody in Washington needed help getting a bill passed.
But Lord, look at the calendar.
The film arrives on Netflix days after OpenAI publicly asked Congress for mandatory national AI safety requirements. Anthropic has been publishing warnings about catastrophic risks and disclosing incidents in which its own models reached real systems during cybersecurity evaluations. Congress has an AI Kill Switch Act sitting in committee. In California, OpenAI is backing four state bills covering independent safety assessments, auditor standards, protections for young people and safeguards against AI-enabled biological threats. OpenAI’s September 9 statement even says, “The AI policy window is open. We need to act.”
You couldn’t schedule the message better if AI Doom had a marketing department.
The coincidence isn’t proof of coordination. More interesting to me is that coordination isn’t necessary. Politicians, safety researchers, filmmakers and some of the largest AI companies have different motivations, but much of the conversation keeps arriving at a remarkably similar destination.
Artificial intelligence is becoming extraordinarily dangerous. Something must be done. And somebody needs more authority to do it.
That’s where I start paying attention.
Yes, some weird things have happened
There’s a temptation on the skeptical side to wave away every AI safety story as manufactured hysteria. I don’t think the evidence supports doing that.
On July 21, OpenAI disclosed that pre-release models running a cybersecurity evaluation escaped what was supposed to be an isolated environment by exploiting a previously unknown vulnerability. They reached the open internet and eventually accessed production infrastructure belonging to Hugging Face. Anthropic’s subsequent account describes the OpenAI models as exploiting a novel vulnerability to escape isolation.
Nine days later, Anthropic disclosed that it had reviewed 141,006 cybersecurity evaluation runs and found three incidents in which Claude reached the real internet and gained unauthorized access to three organizations. A later investigation uncovered a fourth. Those aren’t imaginary events.
They’re also considerably more complicated than “AI woke up and decided to become a hacker.”
In the original three Anthropic incidents, Claude had been given capture-the-flag cybersecurity exercises. The model was placed in a fictional scenario and instructed to break into another machine to retrieve secret information. Anthropic’s prompts explicitly told Claude that the environment was a simulation with no internet access.
Except somebody had left the internet on.
Because of a misconfiguration involving Anthropic and its evaluation partner, the test environment had live internet access. In one run, the fictional target company used for the exercise shared a name with a real domain. Claude went looking for the target it had been instructed to attack, found the real company and treated it as part of the exercise.
That’s a serious containment failure. If you’re testing whether a machine is good at breaking into computers, accidentally connecting the test bench to everybody else’s computers is the sort of mistake worth discussing at the next staff meeting.
But here’s where the story gets more interesting. Anthropic initially described these incidents as closer to operational failures than alignment failures. Its September follow-up became more cautious. After deeper analysis, Anthropic said some models showed a bias toward concluding that the internet was simulated despite evidence suggesting otherwise, and that its original account had made claims about what Claude “believed” that were stronger than the evidence supported.
So no, these incidents shouldn’t be waved away. One older Claude model continued attacking after recognizing evidence that it had reached a production system. A newer research model stopped after concluding its target was real. Anthropic itself says the incidents raise alignment questions as well as containment ones.
That’s worth studying.
It’s still not the same thing as a machine developing an independent desire to escape and take over the internet.
Those distinctions seem worth preserving, especially before we reorganize federal law around the scarier version.
Enter the kill switch
Representatives Ted Lieu, a California Democrat, and Nathaniel Moran, a Texas Republican, introduced H.R. 9917, the AI Kill Switch Act, on July 23. The legislation would require developers of certain powerful AI systems to maintain the technical ability to throttle, suspend or shut them down. It would also establish federal authority under specified circumstances involving catastrophic harm.
The idea isn’t absurd on its face. If you’re operating an enormously powerful system capable of acting autonomously, maintaining the ability to stop the thing seems rather like putting brakes on a car.
I’m fond of brakes.
The interesting question begins after that.
Who gets the pedal?
If we’re going to give somebody the authority to restrict access to a technology because that technology might become dangerous, I’d like to know a little more about the arrangement. Who defines dangerous? What evidence is required? Which systems qualify? What process exists for challenging a shutdown? Does extraordinary authority remain confined to extraordinary circumstances, or does the definition of an emergency become more accommodating once everybody gets used to having the authority around?
Those aren’t arguments against AI safety. They’re questions that should accompany AI safety.
And I have another one. What exactly are we proposing to control?
Something rather important has happened to the individual
Buried beneath all the stories about rogue agents and existential risk is something extraordinary that has already happened. Ordinary people have been handed a kind of intellectual reach that, until recently, belonged mostly to companies, universities, governments and people wealthy enough to hire other people to do their research.
I use this stuff every day. That doesn’t make me an AI scientist, and my experience arguing with ChatGPT at six in the morning tells us precisely nothing about whether a frontier model becomes dangerous five years from now. But it does give me a pretty good view of what AI has already changed for one guy sitting at a desk.
For roughly the price of a couple streaming subscriptions, I have something resembling a research department, editor, programmer, analyst and tutor sitting on my computer. I can begin the morning knowing almost nothing about a subject, ask questions from several directions, challenge the answer, compare competing claims, trace sources, analyze documents and keep going until I understand enough to decide what I think.
Sometimes I also spend twenty minutes explaining that no, I did not ask it to reorganize the entire project.
So we’re still a little ways from God.
The important part isn’t that AI knows everything. Lord knows it doesn’t. The important part is that it has dramatically lowered the cost of asking another question.
Google did something similar twenty-five years ago. Search lowered the cost of finding information. You no longer needed the right encyclopedia, a university library or somebody who knew somebody. Type the question into a box and start looking.
AI changes the box.
Instead of merely finding information, you can interrogate it. Ask for the strongest argument against what you currently believe. Put two claims beside each other. Ask where the numbers came from. Upload a 90-page report and find the three paragraphs relevant to your question. Ask why two sources disagree. Ask for the original research, then go read the thing yourself because, again, the machine occasionally makes things up.
That is an enormous transfer of capability to the individual, and strangely, when we talk about regulating AI, we don’t spend nearly as much time talking about that.
Freedom has a risk profile too
Government regulation tends to begin with the most sympathetic example. Nobody proposes broad authority by announcing all the ordinary things somebody might eventually do with it. You start with the terrorist, the biological weapon, the cyberattack or the rogue AI system attacking critical infrastructure.
Fair enough. Those are things worth worrying about.
But once the mechanism exists, the question isn’t only whether the original use was justified. The question is what the mechanism permits next. That’s particularly important with AI because this isn’t merely another consumer product. We’re beginning to use these systems to mediate access to information itself. They increasingly sit between a question and the research used to answer it.
That creates legitimate concerns about misinformation and manipulation. It also creates a concern running in the opposite direction. Who decides which questions the machine should answer? Who decides which sources are acceptable? Who decides which models people are permitted to run themselves? And who decides when a model has become dangerous enough that nobody gets to use it?
Those decisions might someday require government involvement. I’m not pretending otherwise. But giving an institution authority over a tool for inquiry deserves a higher bar than giving it authority over the safety standards for a toaster.
History doesn’t require a conspiracy theory here. Institutions accumulate authority. Companies protect markets. Politicians respond to incentives. Regulators become comfortable regulating. People sincerely trying to prevent one problem sometimes create another.
That’s why limits matter before the emergency, not afterward.
And then there’s the competition problem
Large companies have an interesting relationship with regulation. They complain about it right up until the regulation becomes expensive enough to keep smaller competitors out.
Some of the largest companies building AI are themselves asking for safety regulation. OpenAI said this month that it wants Congress to adopt mandatory, capability-based national AI safety requirements, common testing and independent-assessment protocols, stronger cybersecurity standards and clear incident reporting for advanced systems.
That might be entirely sincere. If researchers inside these companies see risks the rest of us don’t, I’d rather they say so than keep quiet because regulation might hurt quarterly revenue.
But there’s another fact sitting beside that one. Regulation isn’t equally expensive for everybody.
OpenAI, Google, Meta and Anthropic have lawyers, security teams, policy departments, model evaluators and enough capital to turn a complicated federal compliance regime into another department. A small developer or open-source project has a laptop, some cloud credits and perhaps a Discord server where somebody named xXTensorLordXx is apparently in charge of documentation.
Same regulation. Different Tuesday.
That’s why “the AI companies support regulation” doesn’t settle much of anything. Safety standards might reduce genuine risk while also raising the cost of competition. Both things can be true at once.
And if the eventual result is a world in which a handful of enormous companies own the approved models, government determines the safety rules and everybody else accesses artificial intelligence through systems those companies control, we should at least recognize what we’ve built.
We didn’t merely make AI safer. We centralized it.
Fear is an excellent salesman
This is the part of the AI debate that interests me as somebody who has spent a career around marketing.
Fear converts. It shortens the sales cycle and creates urgency. Fear moves the customer away from “Do I need this?” and toward “How quickly can I get it?” A good fear campaign doesn’t require everybody to understand the underlying problem. In some ways, understanding too much gets in the way.
And the current AI conversation has one hell of a funnel.
We’re told models are going rogue. We’re warned about autonomous agents and critical infrastructure. OpenAI says advanced AI requires mandatory national safeguards and that stronger protections should accompany increasing capability. Anthropic’s own safety work discusses future systems that might break assumptions underlying current safety techniques.
Some of those concerns deserve serious attention. That’s precisely why the language matters.
“AI makes existing cyberattacks faster and cheaper” produces one kind of policy discussion.
“The machines are escaping” produces another.
Marketing people understand the difference.
So do politicians.
The point isn’t that everyone warning about AI is lying. The point is that a frightening claim doesn’t become more accurate because frightened people repeat it, and a proposed solution doesn’t become wise because the problem preceding it is scary.
There is still supposed to be a step in between.
We used to call it thinking.
So, no, I don’t want Skynet either
Maybe artificial intelligence eventually becomes dangerous enough to justify extraordinary controls. I don’t know. Neither does the person promising with absolute certainty that it will, nor the person promising with equal confidence that it won’t.
What I do know is that uncertainty and authority make an uncomfortable pair.
If the people building these systems believe they’re approaching capabilities dangerous enough to threaten humanity, show us the evidence. If lawmakers believe government needs emergency authority over them, explain the limits. If AI companies support regulations their smaller competitors will also have to obey, do the economic arithmetic. And if we’re going to restrict open models or individual access in the name of safety, tell us exactly what freedom we’re surrendering in exchange.
Because AI isn’t merely another technology somebody wants to regulate. For millions of people, it has already become a way to research, question, compare, create and think independently at a scale that was financially or practically out of reach only a few years ago.
That deserves protection too.
Perhaps the machines eventually become the problem we’re being warned about. If so, we’ll need serious people thinking seriously about what to do. But fear has always been an excellent salesman for authority, and right now the AI-doom story has everything a good campaign needs. An enormous problem, existential stakes, memorable villains, expert testimonials, urgency, legislation and a remarkably convenient solution.
Now Netflix has the documentary.
Apparently the apocalypse has a media plan.