Mr. The Plague here. October 2026.
I keep hearing the same three claims about an AI pentest, usually from people who are not on the engagement and are not driving the model. AI cannot find real bugs. AI red teaming is theater. Agents are about to replace the tester. None of those match what I am seeing on scoped work this fall.
As of October 2026, AI is not a replacement for a pentester, and it is not theater. It is an augmentation: more coverage, more depth, higher confidence, still gated by someone who already knows where issues live.
If you are not running the tests, you are not qualified to call them either.
01 The three rumors
I am not arguing with a cartoon. These are the claims as I have actually heard them.
“AI can’t find real bugs.”
False. On current tests I am finding previously undiscovered complex chains in both hand-rolled and COTS software. Misconfigurations show up anywhere, and the models are genuinely good at parsing, aggregating, and correlating large datasets.
That is the job a lot of the time. Not inventing a new bug class. Not missing the one that was already in the traffic.
“AI red teaming is theater.”
False, with a caveat. Getting a model to do the work can be hard, and you still have to understand the layers of whatever you are building. Frontier models will generate complex shellcode, implants, phishing sites, and red-team server components if you know what you are asking for.
Knowing what to ask for is the skill. A prompt is not a substitute for it.
“Agents replace pentesters.”
False, as of October 2026. You still need a deep model of how and where issues appear in web apps and networks. If you have that, you can point the system at roughly ten times the surface you could cover alone. If you do not have that, the agent will confidently waste the window.
02 Who is actually saying this
The FUD I have heard about an AI pentest is mostly from people who are not in pentesting and are not using AI on real work. They are reacting to demos, vendor slides, or a screenshot of a model refusing a jailbreak.
That is not an engagement.
I will grant two real failure modes, and they are not the same thing as “AI cannot test.”
- A scanner with a chatbot stapled on.
- A tester who pastes model output into a report.
Both exist. Neither is an argument that the work cannot be done.
Do not argue with a strawman. Argue with the claim as you heard it.
“It cannot find bugs” is a different sentence from “some products are slop.” I have seen both. Only one of them is true as a blanket.
03 What a useful AI pentest looks like
Useful does not mean full automation. It does not mean the agent closed the engagement. It does not mean I walked away and came back to a PDF.
A useful AI pentest means more coverage, more depth, and more confidence on a scoped test, with a human still deciding what is a finding. The model proposes. I reproduce. If I cannot reproduce it, it does not exist.
Amazing augmentation. Not a replacement. Not ready to run alone.
Everything below is how I keep that sentence true on a client job.
04 The engagement shape
An AI pentest is never one shot. Multiple models, multiple tools, multiple prompts, multiple phases. Anyone selling a single prompt that “does the pentest” is selling a demo.
Phase 0 is a plan, not a scan
A high-end model is used only to define the test. What will be tested. What I am looking for. How deep. How destructive. What to avoid. That phase does not touch the target. If the plan is wrong, every later phase is wrong faster.
Then the tools get connected
Burp, through an MCP AI bridge, so most of the traffic is available for audit and later analysis. SquidC5’s MCP server for OAST, shells, staging, and anything else that needs an external server. The model is not browsing a mystery box. It is looking at the same proxy history and the same callback infrastructure I would use by hand, just across more of it.
Each phase is time-boxed
Test X for N minutes. Report only what reproduces. Every finding needs the request, the response, and the steps to recreate, including curl. If it cannot be replayed from those artifacts, it is a note, not a finding.
Reporting is its own phase. It is not a dump of the agent log. The log is how I check the model. The report is what a client can act on.
05 What I will not hand the model
Anything that posts to the server in a way that costs the business money, or that pulls an out-of-scope person into scope, does not go to the model. It usually does not go to a human tester on autopilot either.
A customer contact form is the concrete example. I do not need the model blasting that, and I do not need a person blasting it.
One careless submit can page a salesperson, create a ticket, or email a customer. That is not a finding. That is a rules-of-engagement failure.
The model does not get to infer the rules. If the action has a side effect outside the test, it is named as excluded before the phase starts, or it is not in the phase at all.
06 Scoping is explicit or it is wrong
Do not let the model infer depth, destructiveness, or exclusions. Prompts and instructions name them. “Be careful” is not a scope.
This is a scope:
- Do not submit forms that notify a human.
- Do not delete data.
- Do not follow links off this host.
- Stop at proof. Do not exfiltrate.
Up front, define the expected scope of every test user. Roles. What each user is allowed to see. What shared access is by design. Write it down before the first request, not after the first false positive.
If you skip that, broken-access-control testing turns into a false-positive factory. The model is not reliable at deciding “by design.”
It will see user A’s identifier in user B’s response and call it an IDOR, including when the product is a shared inbox on purpose. That judgment is yours. Give it the facts before it starts, or you will spend the triage window teaching it the org chart.
07 Where it has actually moved findings
IDOR is the surprise. DeepSeek and Grok have been unusually good at watching responses, finding API endpoints, and crossing streams between users. Not in a demo app. On real tests, in both custom code and COTS, including chains I had not already written down.
Broken access control sits next to that. This was already the class of bug I liked best, because it is where the business actually leaks. The gain is not a new bug class. It is how fast and how thoroughly those chains get traced once the traffic is in front of a model that will not get bored on response forty.
I am not going to borrow a vendor metric for this. The claim is narrower: on the engagements I am running in 2026, this is where the augmentation shows up in the report, after I have reproduced it myself.
08 What a finding has to look like
Verbose. Every step the model took. All evidence. The request, the response, the recreate steps, the curl. If the writeup is a paragraph of confidence and no artifact, I throw it out.
I independently check the audit log. Burp history, callbacks, the phase notes. The model’s story is not the evidence. The evidence is the traffic. If the story and the traffic disagree, the traffic wins.
The prompt has safeguards so it only emits findings that reproduce, with recreate steps. That cuts noise. It does not replace reading.
I still read every one. I never copy a finding into the report.
This is the line between a pentest and slop. A pentest has a human who already knows the bug class, looking at evidence, deciding what is real. Slop is a transcript with a logo on it.
09 The failure that wasted time
Unstated user and role scope. Users who can see each other’s data by design got reported as IDORs. The chains were internally consistent. They were also wrong, because I had not told the model what “allowed” looked like.
Postmortem change: role scope is part of the initial prompt, not something I discover in triage.
- Who the users are.
- What they may see.
- What is shared on purpose.
- What would be a real cross.
If I cannot write that before phase 1, I am not ready to point a model at access control.
Human review after the model finishes is not optional. That is the control, not a nice-to-have.
The model can be fast and still be wrong about intent. Intent is a scoping fact. I own it.
10 Model mix
Every engagement is a mix of agents and models. I do not have a single stack I pretend is the product. I have a rotation that keeps the phase moving when one model stalls.
| When | What I use | Why it is there |
|---|---|---|
| They will do the work | Frontier xAI and DeepSeek | The ones that have been useful on IDOR and access-control chains |
| They refuse | Local Qwen 3.8 27B, abliterated, 3090 Ti, ~34 tok/s | Good enough to keep the phase moving. Not a reason to pretend the frontier model did the step |
| The data is not mine to send | The box I control, or it does not go in a prompt | Client traffic and in-scope credentials stay off a third-party pipe |
Hosted versus local changes what I am willing to put in context. That is the rule I actually use. It is normal handling of client data, applied to a new pipe. I am not going to lecture about it beyond that.
11 Red team, without the theater
Same constraint as web. You have to understand the layers, and model refusal is a real operational problem. If you do not already know what a stage is for, a model that will write one is not an advantage. It is a way to ship something you cannot operate or explain.
What I have gotten frontier models to produce, when the ask is specific and the operator already knows the layer: complex shellcode, implants, phishing sites, and red-team server build-out. I am not going to turn that into a tutorial. Achievable is not the same as appropriate to publish, and it is not the same as unattended.
What I do not claim: unattended operations, or that a junior with a chatbot can run this.
The external-server piece, OAST, staging, shells, is infrastructure. SquidC5 is how I expose that to the model over MCP. It is not a substitute for the person who decides what is in scope and what gets kept.
12 What a buyer should ask about an AI pentest
| Question | Answer |
|---|---|
| Should you buy a pure AI pentest? | No. A model mailed back as a transcript is a scanner with better prose. |
| Should you avoid a pentester who uses AI? | No. If they can show the evidence path, the refusal fallback, and the human who kills bad findings, you are buying coverage. |
| Should you ask how it is used? | Yes. Models, providers, storage, consumption, zero data retention. You already ask where the VPN concentrator lives. Ask where the prompts live. |
Three questions you can steal:
- What is the model allowed to send?
- Who reviews a finding before it hits the report?
- What happens to traffic and prompts after the test?
If the answers are vague, you are not buying a methodology. You are buying a vibe.
13 Close
October 2026, again, so nobody can slide the date. Not useless. Not autonomous. Dangerous only if you skip the human who already knows the bug class.
The danger is not the model finding something. The danger is treating its transcript as the report, or letting it infer rules of engagement you never wrote down.
The people calling it theater are not running the tests. The people selling it as a replacement are not either.
I am running the tests. The model changed what one competent tester can cover on an AI pentest. It did not change who is responsible when the finding is wrong.