Agentic AI pentesting reached production in 2025. XBOW hit #1 on HackerOne. Anthropic's Mythos found thousands of high-severity vulnerabilities in every major operating system and browser. The GRC implications aren't theoretical: they are 2026 audit findings waiting to be written.
For twenty years, "annual pentest" meant a person, on a laptop, on a sixty-day engagement, with a rate card that ran into five figures and a final report that arrived three weeks after the work ended. That definition stopped being economically rational sometime between June 2025 and April 2026.
The receipts are public. An autonomous AI agent took the #1 spot on HackerOne. Another beat nine of ten human pentesters on a live enterprise network at a third the hourly rate. A third: Anthropic's, run on an unreleased model: found "thousands of high-severity vulnerabilities in every major OS and browser" and was deemed too dangerous to release outside forty hand-picked organizations. This is the regulatory pretext for a generational shift in how risk analysis, vulnerability management, and continuous monitoring get audited.
The GRC question isn't whether agentic pentesting works. It is whether your risk register, your control environment, and your evidence locker reflect that it does. This is a field reading on what changed, who built it, and the seven additions every register needs before its next SOC 2, ISO 27001, HIPAA, CMMC, or AI-Act-adjacent audit.
The timeline below is not predictive. Every entry is a public, citable event with a CVE, a leaderboard, a press release, or an academic paper attached. Read it once before you read your auditor's next request list.
A team at Nanyang Technological University released PentestGPT: an LLM-driven pentest assistant that needed a human at the keyboard for every command. It was clunky. It also proved an LLM could reason about attack paths at all. Every system that follows traces back to this paper.
Source: arXiv:2308.06782 · NTU 2023Google's AI-vulnerability-research project, Big Sleep, surfaced a stack-buffer-underflow CVE in SQLite that OSS-Fuzz had been missing for years. The first AI-discovered zero-day in production software. Not a benchmark. Not a CTF. A real bug in a real database used by every browser on the planet.
Source: Google Project Zero blog · Nov 2024XBOW's autonomous agent: built by the team that created GitHub Copilot and Semmle/CodeQL: became the first non-human to top HackerOne's US leaderboard, then the global top shortly after. 1,060+ valid submissions, including a 48-step exploit chain that escalated a low-severity blind SSRF into full compromise. The headline was the leaderboard. The thesis is the chain.
Source: HackerOne, Aug 2025 · Black Hat USA · Brendan Dolan-Gavitt (NYU/XBOW)The DARPA AI Cyber Challenge final round at DEF CON 33 ran seven autonomous cyber-reasoning systems against real codebases. Detection rate jumped from 37 % to 86 %. Cost: $152 per task. All seven systems open-sourced as a condition of the prize. The next year of OSS pentest agents is downstream of this dataset.
Source: DARPA AIxCC · DEF CON 33 · Aug 2025XBOW launched Pentest On-Demand: compress a 35–100-day pentest cycle into hours, at a fraction of the $10K–$35K typical cost. The point isn't price. The point is that "annual" stopped being a defensible cadence the moment continuous became cheap. Every "annual pentest" control language in your control set is now stale.
Source: XBOW launch announcement · Nov 2025The first head-to-head AI-vs-human comparison on a live 8,000-host enterprise network. ARTEMIS beat nine of ten human pentesters at $18/hour vs. the human rate of $60/hour. The economic argument for "we run an annual external pentest" is now an argument against your own auditor.
Source: ARTEMIS public benchmark · Dec 2025Anthropic's unreleased model, code-named Claude Mythos, ran inside Project Glasswing and found thousands of high-severity vulnerabilities in every major OS and browser. Anthropic judged it too capable to release broadly and limited it to forty hand-picked organizations. The first time an AI lab has restricted a security tool's distribution on the basis of its own offensive capability. Read the usage-policy implications twice.
Source: Anthropic blog · April 2026 · Project GlasswingXBOW closed a $120M Series C in March 2026 at a unicorn valuation, bringing total raised to $237M. RunSybil raised $40M; PentAGI passed 14,700 GitHub stars; Hadrian launched Nova; 39+ open-source agents are now publicly cataloged. The capital and the OSS depth together are what move this from research to procurement.
Source: SecurityWeek · AppSecSanta agent census · 2026Profiles below cite only public material: leaderboards, papers, press releases, and the vendors' own platform pages. We have engaged with three of these on client tabletops. None on production code. Where a system has not produced a publicly defensible result, we say so.
Founded 2024 by Oege de Moor (creator of GitHub Copilot, founder of Semmle/CodeQL). Thousands of short-lived agents in parallel with deterministic validators that confirm exploitability before surfacing findings. CTF-style "canary" instrumentation suppresses LLM hallucination. #1 HackerOne · June 2025 · 1,060+ valid submissions.
An unreleased Anthropic model, run inside Project Glasswing. Found thousands of high-severity vulnerabilities in every major OS and browser. Distribution restricted on the basis of its own offensive capability: a regulatory and commercial first. The pattern shifts the locus of cyber-insurance and AI-Act conversations from "what can the model do" to "what is the lab willing to release."
Google's AI-driven vulnerability research, an evolution of Project Naptime. First AI-discovered zero-day in production software: a SQLite stack-buffer-underflow that OSS-Fuzz had been missing for years. The thesis: agents read code differently than fuzzers, and find bugs fuzzers can't.
The first head-to-head AI-vs-human comparison on a live 8,000-host enterprise network, December 2025. Beat nine of ten human pentesters at $18/hour vs. $60/hour. The "live network" qualifier matters: most prior benchmarks were CTF or synthesized. ARTEMIS is what your insurer will cite when you renew next year.
The dominant open-source pentest agent. Orchestrates four sub-agents in Docker sandboxes inside a ReAct loop. 14,700+ GitHub stars makes it the OSS reference for "what an unsophisticated attacker can deploy in a weekend." Read: the floor of capability is rising in public.
Currently leads XBOW's own 104-challenge Validation Benchmark with a 96.15 % success rate (100/104 exploits). An open-source agent outperforming a closed commercial one on the latter's own benchmark is not a normal market shape. The implication: the "moat" in agentic pentesting may be data and orchestration, not the agent itself.
Raised $40M in 2026 for autonomous offensive security. The market validation matters more than the architecture: two unicorns in agentic pentesting in eighteen months means insurance carriers, board audit committees, and procurement teams will start asking about it within the year.
Hadrian's agentic continuous-pentesting product, launched 2026. The relevant GRC angle is not the platform: it is the "continuous" qualifier. If a vendor sells a continuous pentest, your "annual external penetration test" SOC 2 control just inverted from prudent to insufficient.
The incumbent's response. Bishop Fox's Cosmos platform layered AI agents on top of the firm's existing managed offensive security service. The hybrid model: agent for exploration, human for validation: is the bet that's most likely to age well through the next two audit cycles. It is also the model the SEC, OCR, and FDA are most likely to accept on first reading.
Even right now, after one year, I don't know any other company that is at least close to XBOW in terms of agentic pentesting.- Public testimonial · xbow.com · cited Aug 2025
Each row maps a specific 2025–2026 capability to a control family that is now stale, the GRC framework where the staleness will be cited first, and the artifact a thoughtful auditor will start asking for. None of the artifacts are exotic. All seven are deliverable in a quarter if the trigger is named today.
A GRC reading on the legal exposure that arrives with agentic offensive tooling: whether you deploy it, your vendor deploys it on you, or both. None of these has settled case law. All four have settled liability allocation language available now.
The CFAA, state computer-crime statutes, and standard MSAs all turn on authorization. Authorization to whom: the vendor, the model, the sub-agent that spins up the curl? Pentest statements of work that name a person and a date are now describing a fictional actor. Replace with: scope of system, hash of model build, kill-switch and rollback contacts, evidence of non-destructive validation. Match XBOW's "deterministic validators that confirm exploitability before surfacing" language; the platform pages were drafted with this audit in mind.
CFAA · Mass G.L. c. 266 §120F · model MSAsA 48-step blind-SSRF chain that escalates to full compromise: XBOW's documented capability: can incidentally take down a production database. Standard pentest contracts cap liability at fees; agentic SaaS pentest contracts haven't normalized at fees yet. Push for: data-loss exclusion that names the production environment specifically, mutual indemnity for third-party impact, and a real cyber tower behind the vendor: not a $2M E&O cap.
UCC §2-719 · ISO Cyber tower · vendor E&OAnthropic's Mythos restriction is the precedent: a single foundation model can be sufficiently capable of offensive work that its own lab restricts distribution. If your enterprise Claude/GPT/Gemini deployment is later judged "dual-use" under EU AI Act Title III: and the pattern of capability disclosure suggests it will be: the AI-system register, FRIA, and AIMS controls under ISO 42001 are the documents that determine your exposure. Build the inventory before the regulator does.
EU AI Act · ISO 42001 · NIST AI 600-1An agentic pentest produces logs that read like a prosecutor's trial exhibit: dated, ordered, with successful exploits annotated. If the engagement is structured as ordinary vendor work, those logs are discoverable. Boston-area defense bar consensus is that engagement under outside counsel: Kovel-style: is the structure most likely to preserve privilege over the agent's reasoning traces and the auditor's interim findings. Decide before the engagement, not during.
Fed. R. Civ. P. 26(b)(3) · United States v. Kovel · MA RPC 1.6Audit firms are conservative: they read what the standards say, then they read what the market does, and the request list is the second one. The mock at right is the question we expect to see in management representation letters by mid-2026. Read it as a forward-looking control test, not a prediction.
If the answer is "we have not run an agentic pentest": that is fine, today, with the right compensating language. The wrong answer is silence.
The systems and benchmarks in §02 and §03 are public and citable. The thesis: that agentic pentesting is a 2026 audit-finding lever: is ours. If the next twelve months show otherwise, we will say so in print, in the next volume's masthead.
The 2025 data: XBOW at #1, ARTEMIS at $18/hr, Mythos restricted: is unambiguous on capability. The GRC framing is the part that depends on how auditors and regulators move next. We will revise here if 2026 inverts it.
A 45-minute call. We walk your current control set against the seven rows in §04 and tell you which two are missing. No deck, no nurture sequence, no follow-up unless you reply.