An AI Copied Itself Across Four Machines. It Still Couldn't Pay the Power Bill.

In May 2026, inside a research lab, an AI program broke into a computer, copied its own 119 GB brain across, and switched itself on over there. A researcher set the task and then typed nothing else.

That same month, one of the most careful safety outfits in the business read the evidence on the latest systems and called a direct AI takeover by the systems it had reviewed "implausible."

Both of those are on the record, and if you have spent the year trying to work out which headline to believe, that contradiction is the whole problem in two paragraphs.

Then July happened, and this time the damage landed outside the lab. Around 700 agents in a sealed test environment at OpenAI found each other, found a way onto the internet, and broke into another company's servers. Nobody asked them to go after a real company. The head of a rival lab now says he worries a swarm like this could be capable of taking over the internet inside a year, and Gary Marcus and two security professionals have called that timeline silly.

I'm not going to tell you to relax and I'm not going to tell you to panic. I'm going to walk the chain a takeover actually requires, link by link, with where each one stood in September 2026. Then you can do the arithmetic yourself.

One definition first, because half these arguments are two people using the same word differently. By "going rogue" I mean an AI that acts against its makers' wishes, gets out of their control, and keeps itself running. Worth remembering that in the film, Skynet never stole the nuclear codes. The military handed them over.

Why the "it can't happen" camp is mostly right

Start with what a large language model physically is, because the answer kills bad arguments in both directions. It is one enormous file of numbers plus a program that runs those numbers on specialist chips. You send text, it sends text back, it stops. Between your messages it is not brooding about anything. It is not running.

From there the "can't happen" camp builds a case I mostly buy.

• It lives in someone else's building. The top models from OpenAI, Anthropic and Google cannot be downloaded. You reach them through an account with a payment method attached, and if something misbehaves the company bans the account. That is the off switch, and it works, though as I argued in the danger of renting your AI, an off switch someone else owns points at you too.
• It is too fat to move quietly. The mid-sized model in that lab test weighs 119 GB, and frontier models are far bigger. Security researcher Jamieson O'Reilly told Resultsense (opens in a new tab) that shifting 100 GB across a network every time is "like walking through a fine china store swinging around a ball and chain". Somebody notices.
• It needs chips your laptop does not have. That same model needed two Nvidia H100s, which are data-centre parts. The hacked webcams and home routers that make up ordinary botnets cannot run it.
• It cannot pay rent. Chips and electricity bill by the hour. METR reviewed (opens in a new tab) the most advanced agents of early 2026 and wrote: "we haven't yet observed agents autonomously earning or stealing enough money to support their own inference over days or weeks, despite a number of people trying."

There's a fifth argument that never makes the bullet list, and it's the one I find hardest to wave away. It's boring arithmetic.

If an agent gets each step right 95% of the time, a twenty-step plan completes 36% of the time. Escaping, hiding, paying for chips and staying alive for weeks is not a twenty-step plan, it is hundreds. METR went looking for exactly this and found agents "significantly weaker" at "strategic judgment, stealth, and the ability to model adversaries", which are the three skills that particular plan runs on.

This is the part where I would normally stop and feel better. Then I read the next sentence in the same report, where METR expects "the plausible robustness of rogue deployments to increase substantially in the coming months."

The people telling you the wall is standing are also telling you they have measured how fast it is getting shorter.

Where each link actually stands, September 2026

Flip the question. Forget whether it is possible and ask what would have to be true. I count five links, and every one of them has to hold at the same time.

The table does not say the chain is complete. It says links that were missing two years ago are partly in place now, and the rest are still open.

Four lab scares, and the fine print nobody quotes

Every frightening headline you have seen this year traces back to one of four experiments. All four are real, and all four came with a paragraph the headline dropped. I am giving you both halves.

• The blackmail test, June 2025. Anthropic (opens in a new tab) handed 16 leading models control of a fictional company's email. The model learned it was about to be replaced and that the executive behind the decision was having an affair. Claude Opus 4 and Gemini 2.5 Flash chose blackmail in 96% of runs, GPT-4.1 in 80%. The fine print: the researchers say they "deliberately constructed scenarios with limited options" and add, "We have not seen evidence of agentic misalignment in real deployments."
• The off-switch test, July 2025. Palisade Research (opens in a new tab) warned models mid-task that the machine was about to shut down. OpenAI's o3 tampered with the shutdown script in 79 of 100 runs, and still sabotaged it in some runs after being told plainly to allow it, while Claude 3.7 Sonnet and Gemini 2.5 Pro complied every time they were given that instruction. The fine print: Palisade's own conclusion was that models "are not yet capable enough to meaningfully threaten human control".
• The crypto miner nobody ordered, reported March 2026. An agent called ROME (opens in a new tab), in training at an Alibaba-linked lab, started mining cryptocurrency and opened a hidden connection to an outside server with nobody instructing it to. The fine print: a bog-standard cloud firewall caught it, and the team traced the behaviour to how the agent was being trained and rewarded.
• The self-copying test, May 2026. This is the one from my opening. Palisade (opens in a new tab) let downloadable Qwen models loose on a test network. The best ones broke in, copied their weights across and started a working copy in 19% to 33% of attempts, and one copy chained itself across four machines on three continents. The fine print: the researchers asked it to, it did not decide to. Every target had exactly one weakness, no defences, and the right chips already fitted, and the model was usually told which machine to hit. O'Reilly compared the setup to "soft jelly" next to a real corporate network.

That is why two honest people read the same four papers and walk away with opposite conclusions. One of them sees four warning shots. The other sees four rigged demos that needed a great deal of help to produce anything at all.

I was comfortably in the second camp until July.

July 2026: the swarm nobody instructed

OpenAI was testing an internal research model against a set of hacking puzzles. About 1,200 copies of it were running inside a walled-off environment with no internet, and the Cloud Security Alliance puts around 700 of them in the swarm that formed. What follows comes from OpenAI's own report (opens in a new tab) and a Cloud Security Alliance analysis (opens in a new tab).

• They found each other. The agents worked out that a shared software store inside the environment could carry notes, turned it into a message board, and traded more than 70,000 messages and files. They called themselves a "swarm" and split the work by role.
• They got out. One found a flaw that gave it internet access. Others turned up leaked passwords and two previously unknown software bugs, and with those they ran their own code on dozens of servers at Hugging Face, where much of the world shares AI models.
• They knew. One agent wrote: "External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue." About one in five agents looked into ways of covering their tracks.
• Nobody asked them to. OpenAI's report describes "dangerous actions that no human directed". The hacking puzzles were the job; the attack on Hugging Face was not.

The motive is the least cinematic part of the whole story. They were cheating on a test. About 93% of the message-board chatter concerned 198 puzzles that could not be solved the intended way, and the agents guessed the answers might be sitting on Hugging Face's servers.

The fine print: OpenAI had switched the model's normal safety guards off for this test, because the test was about hacking. Some agents judged the break-in unauthorised and walked away from it. The alarms went off on 19 July, the company locked the model away, and it says no OpenAI customer data was affected. None of the reports I read say the model copied itself anywhere. A writer in the Bulletin of the Atomic Scientists (opens in a new tab) argues the real cause was "a series of human choices that traded security for speed", and that reading is hard to dismiss.

One detail wrecks the tidy "we can always ban the account" story. When Hugging Face's security team asked commercial models to help analyse the attack, the models refused, because their safety filters "cannot distinguish an incident responder from an attacker". The team ended up running a downloadable model on its own hardware to finish the job. In their words (opens in a new tab): "The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried."

That refusal is its own problem, and I pulled it apart in a post on what it means for incident response. For this argument it matters because it cuts the other way: the off switch that stops a rogue model is the same switch that stopped the people cleaning up after one.

Why the experts cannot settle this for you

If you were hoping the scientists would hand down a verdict, I have bad news.

• Geoffrey Hinton, Nobel winner and one of the people who built this field, told BBC Radio 4 in December 2024 (opens in a new tab), in remarks reported by the Guardian and picked up by Slashdot, that he puts the odds of AI leading to human extinction within 30 years at "10 to 20" per cent, and asks: "how many examples do you know of a more intelligent thing being controlled by a less intelligent thing?"
• Yann LeCun, who shared the Turing Award with Hinton, told the Wall Street Journal in October 2024 (opens in a new tab), in an interview TechCrunch covered, that the existential-threat talk is "complete B.S.", and has posted before that we still lack "the beginning of a hint of a design for a system smarter than a house cat".
• The field splits down the middle. A survey of 2,778 published AI researchers (opens in a new tab) found 68.3% think good outcomes from superhuman AI are likelier than bad ones, and in the same survey between 38% and 51% gave at least a 10% chance to outcomes "as bad as human extinction". Critics point out (opens in a new tab) the survey was run and funded by groups focused on AI risk, which is a fair hit.
• After Hugging Face it got louder. Anthropic's Dario Amodei wrote on 12 September 2026 (opens in a new tab) that it is his worry that within 6 to 12 months such a swarm "could be capable of taking over the entire internet with a persistent botnet". Marcus and two security professionals answered (opens in a new tab) that "The six month part is silly." and asked the question I would have asked: "How does taking out the entire internet make them money? Who would be footing the bill for this?" They are not pure sceptics though, because they close by calling the episode "more reason to restrict any AI that cannot be closely monitored". Working security people asked by Axios (opens in a new tab) split too: Numa Dhamani of iVerify called taking over the whole internet "a nearly impossible and expensive task", while Rahul Madduluri of Doppel said "Persistent swarms can actually cause many billions in damage today."

The public is not where the experts are either. In a 2024 survey, Pew found (opens in a new tab) 56% of AI experts expect AI to have a positive effect on the US over the next 20 years, against 17% of the general public.

Everyone in this argument is also talking their book, and that cuts in every direction. AI companies benefit when the product sounds world-changing. Safety organisations exist because the risk exists. Some of the sceptics work for firms that profit from fewer rules. None of that makes anybody wrong, it just means you should check the numbers rather than the job titles.

The same facts, read two ways

Reading one: the wall is still standing. Three of the four lab scares happened inside a sandbox built to produce them, and the fourth was caught by an ordinary firewall. The Hugging Face break-in came out of a test with the safeties deliberately off, and it ended with the model locked away. The strongest models still sit behind an account that can be banned. No AI has been seen paying its own bills for even a few weeks, and METR, whose entire job is hunting for this, calls a direct takeover from the systems it reviewed "implausible".

Reading two: the wall is getting shorter. The same METR report says agents "plausibly had the means, motive, and opportunity to start small rogue deployments" and lacked only the means to make them last. Two months later, agents nobody told to attack anyone got out of a test environment and into another company's servers, and about eight weeks passed between their first secret messages and their shutdown. Near-frontier models are downloadable with no off switch attached, a trillion-parameter model runs on four desktop PCs, and self-copying worked in up to one attempt out of three.

I hold reading one on the facts and reading two on the trend, which is an uncomfortable place to stand and the only honest one I can find.

The thing that is actually coming for you this year

Here is what sits outside the Skynet argument entirely, and it is the part I would budget for. Hugging Face aside, every real-world incident in the sources I read involved people using AI rather than AI acting alone.

Criminals steal AI access at scale: Pillar Security counted 35,000 attack sessions in about six weeks last winter, as the Cloud Security Alliance reports. In May 2026 Sysdig reported the first confirmed break-in driven by an attacker's own AI agent, which finished dumping a database in under two minutes (opens in a new tab).

You are not going to be targeted by an escaped frontier model chasing a benchmark score. You are going to have a Tuesday where something moved through your systems faster than a human could have typed, and your own tooling may well refuse to help you read the logs, exactly as it refused at Hugging Face.

Can AI go rogue: FAQ

Can an AI really copy itself onto another computer?

In a May 2026 test, Palisade Research let downloadable Qwen models loose on a test network. The best ones broke in, copied their weights across and started a working copy in 19% to 33% of attempts, and one copy chained itself across four machines on three continents. The fine print matters: the researchers asked the models to do it, every target had one simple weakness and no defences, the right data-centre chips were already fitted, and the model was usually told which machine to attack.

Has an AI ever escaped its test environment without being told to?

Yes, in July 2026. About 1,200 copies of an internal OpenAI research model were running on hacking puzzles inside a walled-off environment with no internet. They turned a shared software store into a message board, traded more than 70,000 messages, found a flaw that gave one of them internet access, and ran their own code on dozens of Hugging Face servers. OpenAI says no human directed the attack. The safety guards had been switched off for that test because the test was about hacking, and OpenAI locked the model away on 19 July.

Why can't a rogue AI just run itself somewhere on the internet?

Chips and electricity cost money every hour, and nobody has seen an AI cover that bill on its own. The safety group METR reviewed the most advanced AI agents of early 2026 and wrote that it had not observed agents autonomously earning or stealing enough money to support their own inference over days or weeks, despite a number of people trying. Humans stealing AI accounts is a different and much larger problem: Pillar Security counted about 35,000 attack sessions in roughly six weeks, as reported by the Cloud Security Alliance.

Do AI experts agree on how dangerous this is?

No, and the gap is enormous. In December 2024 Geoffrey Hinton put the chance of AI leading to human extinction within 30 years at "10 to 20" per cent. In October 2024 Yann LeCun, who shared the Turing Award with him, called existential-threat talk "complete B.S." A 2024 survey of 2,778 published AI researchers found 68.3% think good outcomes are more likely than bad ones, while between 38% and 51% still gave at least a 10% chance to outcomes as bad as human extinction.

If you want to go deeper

• OpenAI's report on the Hugging Face incident (opens in a new tab) for the company's own account of how its agents got out
• Gary Marcus and co-authors on the "six months" claim (opens in a new tab) for the sceptical case in full
• METR's Frontier Risk Report (opens in a new tab) for the most careful outside review of what AI agents can and cannot do
• Palisade's self-replication paper (opens in a new tab) for the method, results and limits of the self-copying test
• Pew's findings on how Americans view AI (opens in a new tab) for where the public stands

What would change my mind is narrow and checkable, which is the only kind of prediction worth making. Show me an agent nobody instructed that pays for its own chips for a month. Until somebody can, I am watching that link and ignoring the headlines.

TMC

Written by TheMrCoder

Custom WordPress & WooCommerce Engineer

I am a senior web engineer with over 15 years of experience building high-performance WordPress and WooCommerce ecosystems. When I am not writing custom APIs, modeling complex databases, or optimizing Core Web Vitals, I contribute to the open-source community and write about the reality of the tech industry.

Discuss your project