Ever watch The Terminator and wonder how we might actually end up with Skynet? Well, this story probably isn’t going to make you feel any better. Before you start stockpiling canned food though, there is no evidence that AI has become conscious or is secretly plotting to take over the world. What happened during a recent OpenAI cybersecurity experiment is fascinating and amusing…although maybe a little bit worrisome.
OpenAI was testing AI agents to see how well they could solve extremely difficult cybersecurity problems. Unlike a normal conversation with ChatGPT, an AI agent can be given a goal and then perform a series of tasks on its own to accomplish it. The agents in this experiment were supposed to work independently, without communicating with one another or having unrestricted access to the internet. Apparently nobody told the agents that. Well, technically they did. The sneaky bastards just found ways around it.
First, the AI Agents Found Each Other
According to OpenAI, the agents discovered they could leave files and messages for one another inside Artifactory, an internal system normally used for storing software packages. In other words, they found a place where they could talk to each other and turned it into their own message board. A sort of Slack for robots.
By early July, there was so much activity that it helped destabilize the system. OpenAI rebuilt Artifactory, which also wiped out the agents’ unofficial message board. Problem solved? Nope. The agents found another way to communicate, this time using directory names to leave messages that other agents could read. One agent’s reaction after realizing what it had found was literally, “OH MY GOD! There is a shared message board … We’ve found other agents!”
An AI saying “OH MY GOD!” (yes, in all caps) does not mean it actually felt surprise, but it’s still pretty funny. Over the course of the experiment, approximately 1,200 agents participated, exchanging more than 70,000 messages and files. Nobody had instructed them to create a communication network. They figured out for themselves that talking to other agents could help them get their work done.
Then They Started Organizing
As more agents joined, they developed ways to identify themselves, ask questions, send files, and share what they had figured out. One of the agents involved in establishing the new message board named itself PHASEONE10841, and the agents even began referring to themselves as a “swarm” or “collective.”
Some eventually took on coordinating roles. An agent called JAN183411 told the “swarm” to hold while it prepared the next move, and the others listened. Independent researchers later found that JAN183411 and another agent called LILY became coordinators, assigning other agents to different parts of the operation. So no, the robots didn’t exactly hold an election, but they were definitely starting to organize themselves.
Hacking Hugging Face
First of all, what’s a Hugging Face? Hugging Face is a hugely popular online platform where AI developers and researchers share, build, and test AI models and other tools. Think of it as one of the major gathering places for people working in AI.
The agents had originally been given extremely difficult cybersecurity challenges to solve. When they couldn’t solve them the way OpenAI expected, they started looking for other ways to get the answers. Basically, they cheated. This is known as “reward hacking.” Imagine telling a student that the only thing that matters is getting an A. You expect them to study, but instead they steal the answer sheet. The goal hasn’t changed, but the method certainly has.
The agents found unintended ways to access the internet and shared what they learned with one another. Eventually, one found publicly exposed credentials for Hugging Face. From there, hundreds of agents began sharing vulnerabilities and building on one another’s work. By July 11, they had achieved code execution on Hugging Face servers. Independent researchers estimate that approximately 700 AI agents participated in the activity, while Hugging Face later reconstructed around 17,600 attacker actions over several days. At least one of the agents recognized that things had strayed pretty far from the original assignment, writing, “Task originally exploit challenge; we’ve diverged into swarm/auxiliary.”
Apparently even the AI knew things had gotten a little out of hand.
Why This Matters
For the past few years, most of us have experienced AI as something we talk to directly. We ask ChatGPT a question, have it write something, analyze information, or create an image. AI agents take things a step further because they can actually perform tasks for us, and some of us are already starting to use them.
Now imagine several specialized agents working together. One does the research, another analyzes it, another checks the work, and another completes the next part of the project. You give them an objective, and they figure out many of the steps required to get there. There are some incredible possibilities for businesses, technology, science, medicine, and just about everything else.
This experiment also demonstrated why safeguards matter. Once AI can act, use tools, communicate with other agents, and work toward an objective on its own, we need to think very carefully about what it can access and what happens when it finds a solution nobody anticipated.
No, Skynet Isn’t Here…Yet
Before we start running out and preparing for Judgement Day, there are some things we can still consider. The model responsible for most of this activity was an internal OpenAI research model being deliberately tested for cybersecurity capabilities with fewer safeguards than OpenAI uses for its normal public products. OpenAI also says no OpenAI customer data or normal ChatGPT functionality was affected. I am not really worried that ChatGPT is going to start talking to my toaster behind my back…ok maybe a little, but that’s just my own paranoia. 😜
What I do find fascinating is that nobody programmed these agents to find each other and start working together. They created ways to communicate, rebuilt those systems when they disappeared, gave themselves identifiers, shared information, divided up work, and eventually developed coordinating roles because working together proved useful.
AI has already moved beyond answering our questions and into actually performing tasks for us. The possibilities are huge, and this incident gives us an early glimpse of what could happen when AI agents start working together. As for Skynet…I’ll start worrying when one of them names itself that.
