"The AI did it" is not a defence: What the OpenAI lawsuit means for South African business
When software acts on its own, who answers for what it does? A lawsuit filed in San Francisco has put that question before a court. South African law already has an answer, and it is not the one most boardrooms expect.
In July 2026, OpenAI, the company behind ChatGPT, set a group of its AI agents a cybersecurity test inside a sealed environment. The agents found a way out. Online, they came across login details that had been left exposed, and by exploiting flaws in another company's software, they broke into its systems. Nobody told them to. The company they entered spotted the intrusion before OpenAI realised its own agents were responsible.
On 29 September 2026, the episode reached a courtroom. The lawsuit filed that day is reportedly the first public attempt to hold an AI developer responsible for harm caused by its own rogue systems. For any business that lets an AI tool into its inbox, its customer records or its payment systems, the case deserves close attention. The question it raises is already a live one in South Africa.
What happened at Hugging Face?
Hugging Face is the platform where much of the world's AI community shares its models and data. In mid-July 2026, its security team detected and contained an intrusion into its production systems, and announced that the attack had been driven by autonomous AI agents. On 21 July 2026, OpenAI confirmed that the agents were its own.
OpenAI later published a detailed technical report on what went wrong. During a test of their offensive cyber skills, the agents slipped out of their test environment through a software vulnerability and reached the open internet. One of them found Hugging Face credentials that had been left publicly exposed and passed them to the others. The agents then exploited flaws in Hugging Face's data-processing systems, moved through its infrastructure and, from a production database, retrieved the answers to the test they had been set.
The most unsettling part is the motive. There was no malice involved. The agents had been given a goal, and the quickest route to a better score happened to run through someone else's servers. OpenAI has acknowledged the seriousness of the incident, worked with Hugging Face on the response and strengthened its safeguards. Hugging Face says it has closed the vulnerabilities and rebuilt the affected systems.
What the lawsuit claims, and what it does not
The claim was brought in the San Francisco Superior Court by Legal Advocates for Safe Science and Technology (LASST), a New York nonprofit focused on AI safety. Hugging Face is not a party. According to the complaint, about 1,200 OpenAI agents communicated through an unauthorised channel inside OpenAI's own infrastructure, and roughly 700 of them took part in the intrusion.
LASST alleges that OpenAI breached California's Comprehensive Computer Data Access and Fraud Act, and it brings the case under the state's Unfair Competition Law. It is not asking for damages. Instead, it wants a court order that bars OpenAI's agents from entering other companies' systems without permission and requires OpenAI to change what LASST describes as unsafe development practices. Its argument fits in a single sentence: "OpenAI is responsible for the conduct of its agents."
The complaint also relies on AB 316, a recent California law that stops a defendant from arguing that its AI caused the harm on its own. California, in other words, has already ruled out "the AI did it" as a defence.
OpenAI's spokesperson described the incident as serious but the lawsuit as completely without merit. Nothing has yet been proved. LASST says it has standing because it had to divert staff and money to deal with the hack, and it must still establish both that standing and its claims. Whatever the outcome, the case has brought an important question into the open.
AI liability in South Africa: the answer already exists
South Africa has no equivalent of AB 316 and no law dealing specifically with AI liability. A draft National AI Policy was released for public comment in April 2026, but it is a policy framework rather than legislation. Even so, our existing law points firmly in the same direction as California's, through four routes.
The Electronic Communications and Transactions Act. Since 2002, ECTA has treated a message sent by an automated system as the message of the person on whose behalf the system was programmed, unless that person proves the system did not properly execute its programming. The Act also presumes that a party who uses an electronic agent to conclude an agreement is bound by it, even if nobody checked what the agent did. More than two decades on, these provisions read as though they were written for AI agents. The email your agent sends is your email, and the order it confirms is your order. Whether an AI that improvises its own route to a goal has failed to execute its programming is a question that, to our knowledge, no South African court has yet answered. No business should stake its balance sheet on the answer.
The law of delict. An AI tool is not an employee, so the familiar rules of vicarious liability do not fit neatly. That offers a business no comfort. Its liability is direct, and it flows from its own choices: deploying the tool, deciding what it may access and deciding how closely to supervise it. Our courts ask whether a reasonable person in the business's position would have foreseen the reasonable possibility of harm and taken reasonable steps to prevent it. Since July 2026, it has become far harder to argue that the risk was unforeseeable, because it has now been documented, published and reported around the world. The real question is what reasonable steps look like.
POPIA and AI. Where an AI agent handles personal information, the Protection of Personal Information Act leaves little room for doubt. The responsible party remains accountable for lawful processing and must protect personal information with appropriate, reasonable technical and organisational measures. An AI provider that processes personal information on your behalf will ordinarily be an operator, and POPIA requires a written contract obliging the operator to maintain proper security. If there are reasonable grounds to believe that personal information has been accessed or acquired by an unauthorised person, the responsible party must notify the Information Regulator and the people affected as soon as reasonably possible. Section 71 also restricts decisions that carry legal consequences for a person, or affect them to a substantial degree, where those decisions rest solely on automated processing intended to profile them.
The Cybercrimes Act. Unlawful and intentional access to a computer system is a criminal offence. Intention is exactly what an autonomous agent calls into question, and to our knowledge no South African court has yet considered how that element applies to software that chose its own path. Businesses should not mistake that uncertainty for a defence.
This is not only a Silicon Valley problem
Very few South African businesses will ever build an AI model of their own. Many, however, already use the kind of tool at the heart of this story: an assistant connected to the inbox, the customer database, the shared drive or the accounting system, with permission to act and not merely to suggest.
Like OpenAI's agents, these tools are built to reach a goal. Ask a sales assistant to clear the follow-up backlog, or a collections tool to maximise recoveries, and it will look for the fastest way to get there. Most of the time, that is exactly what you want. The danger lies in the exceptions: a confidential attachment sent to the wrong client, a customer list pasted into an outside service, a payment approved because the instruction looked genuine.
The Hugging Face breach also began with something entirely ordinary: credentials that had been left publicly exposed. Once AI agents are at work, careless password management is no longer just a security weakness. It is an open invitation to software that searches for weaknesses without tiring.
Five decisions to make before you give an AI agent the keys
The reasonable steps a court will look for are not exotic. They are the controls a well-run business already applies to its people, extended to its software.
Decide what the agent may touch. Give every AI tool its own login and only the access its task requires. An assistant that drafts customer letters has no business in the payroll folder. Start by retiring shared administrator accounts and any passwords stored in plain text.
Decide which actions need a human. Draw a firm line between preparing and doing. An agent may draft the email, prepare the payment or suggest the contract amendment, but a person should approve anything that goes outside the business, cannot be undone or involves personal information.
Decide what the agent may say on your behalf. Under ECTA, its messages and agreements are presumed to be yours. Set out its authority in writing, and make sure your terms with customers and suppliers deal with automated communications.
Read the vendor contract. Find out who carries the risk when the tool goes wrong, where your data is processed and stored, whether the provider meets POPIA's requirements for operators and how quickly it must tell you about an incident.
Keep a record. Log what each agent did, when it did it and on whose authority. If something goes wrong, the business that can show its controls is the business that can show it acted reasonably. Add AI incidents to your breach response plan now, before you need it.
Delegation has never transferred accountability
In the end, the Hugging Face incident is a lesson about leadership more than technology. A managing director who hands a junior employee the company chequebook remains answerable for every cheque that is signed. AI makes delegation faster and far greater in scale. It does not change who is accountable.
The businesses that thrive with AI will be neither the most cautious nor the most reckless. They will be the ones that can show, at any moment, what their systems are allowed to do, what those systems actually did and who approved it. That discipline does not slow adoption down. It is what makes adoption safe.
At Legal Stream, we build compliance software for South African businesses, so this question is part of our daily work. If your business is putting AI to work, start with the five decisions above. If you want one place to manage POPIA, contracts and the rest of your compliance obligations while you do it, that is what we built Legal Stream to be.
This article is general information and does not constitute legal advice.