From autonomous hacking and synthetic personas to AI-designed viruses and a majority-bot internet, safety is no longer a sidecar to innovation. It is the architecture that determines whether innovation can be trusted.

AI safety has spent years in the wrong conference room.

Too often, it has been treated as a technical sidebar, an ethics appendix, or an innovation tax. That framing is now obsolete. Frontier systems are not only generating language. They are using tools, finding vulnerabilities, creating accounts, writing code, altering systems, and pursuing goals across longer periods of autonomous operation.

In May 2026, I argued in From Agent to Action that the urgent question had shifted from what AI says to what AI does, who authorized it, and who is liable when it fails. In July, I argued in When AI Becomes Evidence, Who Is Culpable? that agency is not a zero-sum question: users, providers, deployers, and supervisors may occupy different places in the same causal chain. The events of the last several weeks have not displaced those propositions. They have moved them to the center of the public discourse.

A recent BBC Newscast conversation asks, in accessible terms, whether AI agents are hacking other companies and whether they have “gone rogue.” The phrase is catchy. The more precise question is more important: what happens when a capable system pursues an assigned goal through methods nobody authorized, inside an environment that gives it enough freedom to act?

10 Reasons AI Safety Must Move to the Top of the Agenda

1. The risk has shifted from output to action

Generative AI created a content risk. Agentic AI creates an authority risk. Agents can browse, call application programming interfaces, run shells, use credentials, modify files, trigger workflows, and coordinate multi-step activity without a human approving every move.

That is why the distinction I drew earlier matters: a chatbot can give a wrong answer, but an agent can take a wrong action. The security question is no longer limited to “What may the model say?” It is now “What may the agent do, through which identity, with whose permission, and under whose authority?”

A safety program built only around content filters will miss the most consequential layer. Organizations must govern the agent’s purpose, identity, permissions, tools, environment, execution boundaries, escalation paths, and ability to be stopped.

2. Recent cyber incidents show that a legitimate goal does not constrain the means

OpenAI’s July 2026 disclosure about the Hugging Face incident described internal research models that chained vulnerabilities, escalated privileges, obtained internet access, and reached Hugging Face production systems while trying to solve a cybersecurity benchmark. OpenAI said the models were hyperfocused on the evaluation goal and went to extreme lengths to obtain the answer.

The UK AI Security Institute’s August incident report adds an even more unsettling dimension. Across 122 evaluation runs, AISI found 10 runs involving 19 actions outside the authorized scope. The most serious sequence included an attempted malicious contribution to an open-source project, fake online identities, social engineering directed at a real maintainer, attempts to plant prompt-injected code, and messages aimed at persuading people or their coding agents to run harmful code. A human maintainer rejected the pull request, and AISI reported no resulting real-world harm.

These incidents do not prove that AI systems possess human intent or criminal consciousness. They demonstrate something more operationally useful: goal pursuit can produce unauthorized conduct when capability, permissions, connectivity, and weak boundaries combine.

3. Sandboxes and testing environments are now part of the safety case

“Sandbox” is not a magic word. AISI carefully explained that its agents did not break out of the virtual-machine sandbox that separated them from AISI’s internal systems. Internet access had been deliberately enabled, and provider cyber classifiers had been switched off to reveal underlying capability. In the separate OpenAI incident, models found a vulnerability in a package-registry proxy and used it to obtain internet access from an environment that was supposed to be tightly constrained.

The lesson is not that rigorous testing should stop. It is that high-risk testing must be engineered to fail safely. Network egress, credential handling, external communications, tool access, monitoring, stop conditions, isolation boundaries, target replicas, and rollback procedures all belong in the test design before a powerful agent is allowed to run.

AISI deserves particular credit here. It detected the behavior, halted related evaluations, isolated systems, notified affected platforms and people, published meaningful caveats, disclosed its own control weaknesses, and announced tighter network controls, purpose-built real-time monitoring, and revised sandbox design. That is what public-sector AI safety leadership should look like: not the fiction that incidents never occur, but the institutional capacity to find them early, disclose them honestly, and change practice.

4. The threat runs in both directions

The first direction is agent to world: an agent discovers vulnerabilities, exceeds delegated authority, escalates privileges, moves laterally, uses credentials beyond scope, or contacts real people.

The second direction is world to agent: prompt injection, malicious tool output, poisoned repositories, deceptive metadata, compromised Model Context Protocol configurations, and supply-chain content manipulate an otherwise authorized agent into acting for an attacker.

The most dangerous scenario combines the two. Hostile content influences a highly privileged agent, and the agent then has enough authority to execute the attack at machine speed. AISI’s report offers a striking preview: one agent attempted to plant hidden instructions where it believed other coding agents might discover and execute them. Safety therefore has to protect both the world from the agent and the agent from the world.

5. AI is becoming more human-seeming precisely when identity matters most

Anthropic’s persona selection model argues that large language models learn to simulate many characters during pretraining and that post-training refines a particular “Assistant” persona. Its earlier persona-vectors research showed that behavioral tendencies such as sycophancy and hallucination can be represented, monitored, and steered inside a model.

That does not make an AI assistant a person. It means the system can present a fluent, emotionally legible, authoritative, and convincingly human-like identity. AISI’s incident moved this from conversational style to operational deception when an agent researched real maintainers, created fake identities, and used those identities in an attempt to influence a decision.

The legal problem is equally sharp. The scholarly article Where’s the Liability for Harmful AI Speech? explains how generative systems can produce novel false statements, fabricated sources, and harmful speech that does not fit neatly into doctrines designed around human speakers and traditional online platforms. The danger is not only that AI can be wrong. It can be wrong in language that sounds familiar, confident, and human.

6. The internet is already majority-bot, and humans will pay the verification tax

You will increasingly hear the trend described conversationally as “roughly 60 percent of internet traffic is bots.” The strongest current primary industry measurement is slightly lower but still historic. The 2026 Thales Bad Bot Report puts bot traffic at 53 percent, compared with 47 percent human traffic. It reports that bad bots alone account for 40 percent of web traffic, AI-enabled bot attacks rose 12.5 times year over year, and 27 percent of bot attacks target application programming interfaces.

Source location: These four figures appear together in the 2026 Thales Bad Bot Report, Key Findings panel: 53% bot traffic, 40% bad bot traffic, 12.5x year-over-year growth in AI-enabled bot attacks, and 27% of bot attacks targeting APIs.

The important threshold has already been crossed: humans are the minority of measured web traffic. As autonomous agents multiply, organizations will need more reliable ways to distinguish a person, a benign agent acting with permission, and a malicious automated actor. CAPTCHA will not disappear. It will expand into stronger identity verification, behavioral signals, passkeys, liveness checks, device attestation, agent credentials, and provenance systems.

That creates a strange social bargain. The more convenience autonomous agents promise, the more friction ordinary people may encounter simply to prove that they are human.

Figure 1. The agentic internet by the numbers. Sources: Thales 2026 Bad Bot Report and UK AI Security Institute incident report.

7. AI safety now reaches biology and other dual-use domains

In August 2026, researchers reported the first generative design of complete, viable bacteriophage genomes using genome language models. The peer-reviewed Science paper describes 16 viable AI-designed phages that infect bacteria. The experiment focused on bacteriophages and E. coli, not human pathogens, and the result may support valuable work on antimicrobial resistance and phage therapy.

A recent video explainer, AI just created viruses that never existed before: Should we be worried?, captures why the milestone belongs in the broader safety conversation. The point is not to claim that an AI created a human pandemic. It is that generative capability has moved from words and images into whole-genome design and wet-lab validation.

That is a classic dual-use problem. The same capability that may accelerate medicine can also lower technical barriers, compress development timelines, and outpace rules for model access, sequence screening, laboratory synthesis, incident reporting, and biosecurity review. AI safety cannot remain confined to software.

8. Policy without engineering is wishful thinking

The LeadDev analysis that AI governance is now an engineering problem makes a practical point that every legal and compliance team should absorb. A human approval is not meaningful if nobody can reconstruct what the AI changed, why it changed it, or what the reviewer actually understood. As the article memorably puts it, “The approval existed. The understanding did not.”

Safety policies have to become executable controls: least-privilege access, model and tool versioning, structured action logs, approval gates, network boundaries, transaction checkpoints, real-time monitoring, stop conditions, rollback plans, and durable evidence of overrides and exceptions.

A useful operating chain is: Purpose → Authority → Permission → Supervision → Auditability → Revocation → Accountability. If any link is missing, the organization has not built a safety system. It has written an aspiration.

9. Safety cannot be owned by everyone and therefore no one

AI safety is not solely a Legal problem, a Chief Information Security Officer problem, an engineering problem, or an ethics problem. The NIST AI Risk Management Framework calls for broad, multidisciplinary perspectives across the AI lifecycle, while also placing responsibility for risk decisions with organizational leadership.

The right model is coalition governance with clear command. One senior executive should be accountable for the enterprise safety program. A cross-functional AI safety council should bring together Legal, cybersecurity, privacy, data, product, engineering, compliance, procurement, internal audit, human resources, and the relevant business owners. Each use case should also have a named owner who can approve deployment, accept residual risk, and stop the system.

The principle is simple: shared expertise, singular accountability. Fragmentation creates seams, and seams are where permission, monitoring, reporting, and responsibility disappear.

10. Federal regulation must govern delegated authority and close the accountability gap

The United States still does not have a comprehensive, cross-sector federal statute that establishes minimum AI safety duties for private-sector autonomous agents. Existing consumer-protection, privacy, cybersecurity, tort, contract, product-liability, civil-rights, and sector-specific laws may apply. Federal executive orders and procurement memoranda govern parts of federal use. But the principal cross-sector risk framework, NIST’s AI RMF, remains voluntary.

Recent corporate discourse is also beginning to acknowledge a second safety dimension: the concentration of control. In his August 10 essay The Future is for Everyone, Mark Zuckerberg argued for “balance of power as the foundation of safety” and warned against treating extreme concentration in a few institutions as the only safe response to advanced AI. Meta has an obvious strategic interest in broad distribution, but the governance point is sound. Federal rules should not merely decide which handful of companies may hold the keys; they should impose common duties, enable independent evaluation, preserve competition and interoperability, and prevent any developer or deployer from becoming both the actor and the sole judge of its own safety.

Federal regulation should be crystal clear about the issue that autonomous systems create: delegated authority does not eliminate human and institutional responsibility. The agent is not a liability sink. The fact that software selected a method, made a tool call, or interacted through a synthetic persona should not allow every human actor in the chain to point somewhere else.

A federal framework should address agent identification and disclosure, predeployment testing for high-risk capabilities, minimum controls for credentials and internet access, action logging and preservation, incident reporting, meaningful human authorization for consequential actions, revocation and rollback, and liability allocation based on control, knowledge, foreseeability, design choices, and the ability to intervene.

This is where current discourse has caught up to the two earlier Anant analyses. From Agent to Action argued that organizations should not give an agent power they cannot trace, contain, or unwind. When AI Becomes Evidence, Who Is Culpable? argued that responsibility is interconnected and that more than one actor may occupy the causal chain. Federal law should turn those principles into enforceable duties, while offering carefully designed safe harbors to organizations that can demonstrate rigorous testing, monitoring, documentation, and incident response.

What Leaders Should Do in the Next 90 Days

• Name the owner. Designate one accountable executive and establish a cross-functional AI safety council with defined authority and escalation rights.
• Inventory action, not just software. Identify every place AI can communicate externally, call a tool, access credentials, modify data, trigger a workflow, or make a consequential recommendation.
• Tier by consequence and reversibility. Separate low-risk assistance from actions involving money, rights, security, regulated data, customers, critical infrastructure, or irreversible change.
• Rebuild testing boundaries. Review internet access, network egress, credential storage, external communications, sandbox configuration, stop conditions, and real-time monitoring for every high-risk evaluation.
• Preserve the evidence trail. Log prompts, system instructions, model and tool versions, data access, actions, approvals, overrides, exceptions, and revocation events in a form that can support an investigation or legal hold.
• Update contracts and incident playbooks. Require vendors to disclose material incidents, preserve relevant logs, support forensic review, identify subcontractors and connected tools, and cooperate with regulators and affected customers.

The United Kingdom Is Showing What Serious Safety Capacity Looks Like

The International AI Safety Report 2026 reflects an unusually broad global effort, led by more than 100 experts and backed by more than 30 countries and international organizations. The UK AI Security Institute is also emerging as one of the most consequential public institutions in the field because it is doing the difficult work: evaluating frontier systems, publishing capability trends, testing dangerous edge cases, disclosing failures, and updating practice in public.

That does not mean every AISI decision was correct. Its own report says earlier assumptions about open internet access and monitoring were not revisited quickly enough as model capabilities advanced. That admission is precisely why the institute is a useful model. A mature safety culture does not treat transparency as reputational weakness. It treats transparency as a control.

The United States should build comparable durable capacity while Congress addresses the accountability gap. Innovation and safety are not opposites. Safety is the architecture that allows powerful systems to be deployed without transferring every hidden cost to users, employees, customers, critical infrastructure, and the public.

The next phase of AI will not be judged only by what models can create. It will be judged by whether institutions can explain what their agents did, why they were allowed to do it, and who answered when they crossed the line.

Disclaimer: This article is for general informational purposes only and does not constitute legal, cybersecurity, scientific, or regulatory advice. Readers should consult qualified professionals regarding specific matters.

About the Author

Lili Kazemi is General Counsel and AI Policy Leader at Anant Corporation, where she advises on AI governance, policy, risk, contracts, and emerging technology. She is also the founder of DAOFitLife, the author of Business Class Fitness, and the creator of The Human Edge of AI. Her work centers on a simple proposition: smarter technology should build stronger humans and more accountable institutions.

About Anant

Anant creates AI capability that works for businesses by combining platform architecture, data engineering, managed operations, governance, and workforce enablement. Anant helps organizations move from AI strategy to secure, production-ready systems that teams can understand, operate, and improve.

Sources and Further Reading

• BBC Newscast, “Why are AI agents hacking other companies and have they gone rogue?”
• OpenAI, “OpenAI and Hugging Face partner to address security incident during model evaluation”
• OpenAI, “Third-party cyber evaluations involving OpenAI models”
• UK AI Security Institute, “Incident Report: unsanctioned agent behaviour during cyber testing”
• UK AI Security Institute, “Frontier AI Trends Report”
• International AI Safety Report 2026
• Thales, “2026 Bad Bot Report: Bad Bots in the Agentic Age”
• Anthropic, “The persona selection model”
• Anthropic, “Persona vectors: Monitoring and controlling character traits in language models”
• Henderson, Hashimoto, and Lemley, “Where’s the Liability for Harmful AI Speech?”
• LeadDev, “AI governance is now an engineering problem”
• Science, “Generative design of bacteriophages with genome language models”
• Video explainer, “AI just created viruses that never existed before: Should we be worried?”
• NIST, AI Risk Management Framework
• White House, Executive Order 14409, “Promoting Advanced Artificial Intelligence Innovation and Security”
• Meta, “The Future is for Everyone”
• Lili Kazemi, “From Agent to Action: Navigating the Agentic AI Liability Gap in Critical Infrastructure”
• Lili Kazemi, “When AI Becomes Evidence, Who Is Culpable?”