Disclosure, up front
We write software for clients and we use AI assistance doing it, so this article describes our own tooling. Everything below applies to us. The questions in section 05 are ones we would expect a client to ask us, and we have written them the way we would want to be asked rather than the way that would be easiest to answer.
If you have commissioned software in the last year, some of it was written by AI. Your developer may not have mentioned it, because from their side it is a tool like an editor or a compiler, and mentioning it would be like mentioning autocomplete.
From your side there is one difference worth understanding. AI assistance makes code arrive much faster, and the security defects arrive faster too — disproportionately so. The gap between those two rates is the entire issue, and it is not solved by asking whether your developer uses AI. It is solved by asking what happens after.
What the testing shows
- Roughly a quarter to nearly half of AI-generated code samples carry a security flaw, depending on the study: Veracode's spring 2026 testing across 100+ models found 45% introducing an OWASP Top 10 vulnerability; a separate 2026 study of 522 samples across 6 models found 25.7%.
- Speed and defects do not scale together. Research across large enterprises found AI-assisted developers committing at 3–4x the rate of peers while introducing security findings at about 10x.
- The dangerous part is the confidence. Studies have found developers using AI assistants wrote less secure code while rating their own insecure solutions as secure.
These are vendor and academic studies of models under test conditions, not measurements of your developer. Read them as evidence about the raw output before review, which is exactly the thing review exists to catch.
01Why the numbers vary so much, and which to believe
You will see AI code vulnerability rates quoted anywhere from 20% to 45%. The spread is not sloppiness; it reflects genuinely different tests — different models, different task types, different definitions of a vulnerability, and crucially whether the test measures raw first output or code after any review.
Two things are consistent across all of them, and those are the ones to act on:
- The rate is high enough to matter. Even the low end means a meaningful fraction of generated code has a flaw in it before anyone looks.
- Nobody claims review does not help. These studies measure output, not process. Every one of them is an argument for the review step, not against the tool.
Which is why "does your developer use AI" is the wrong question. The answer is almost certainly yes, it does not distinguish a good supplier from a bad one, and a developer who says no may simply not be telling you.
02The finding that should actually worry you
Not the vulnerability rate. The false confidence.
Research has repeatedly found that developers with AI assistance produce less secure code and rate their own insecure solutions as secure. That combination is worse than either half alone, because the usual safety net in software is a competent person feeling uneasy about something. Code that arrives looking finished, formatted correctly and confidently explained does not trigger that feeling.
The same applies to the 3–4x commit rate against 10x security findings. Volume defeats review before quality does. A reviewer who could carefully read a week's output can no longer carefully read what now arrives in a day, and the review silently becomes a skim.
What this means for a fixed-price build
If a supplier's speed has increased and the price has not, that time went somewhere. Frequently it went into delivering more scope for the same money, which is good. Sometimes it came out of testing and review, which is where you would never see it until later. Ask what proportion of the effort is review and testing — not to catch anyone out, but because a supplier who has thought about it will have an answer and one who has not will find the question surprising.
03Where AI-written code actually goes wrong
The failures are not exotic. In roughly the order we see them:
- Input handling. Generated code tends to assume well-formed input. Injection flaws are the classic result, and they remain the most common serious defect in web applications.
- Authentication and access checks. A generated endpoint often does the thing it was asked to do without checking whether the caller was allowed to. This is how one customer ends up able to read another's records.
- Secrets in the wrong place. Keys and credentials committed into code or config, because the fastest working example puts them there.
- Outdated or invented dependencies. Suggestions reflect what was common in training data, which skews old; occasionally a suggested package does not exist, which is its own supply chain risk.
- Error handling that reveals too much. Stack traces and database messages surfaced to users, because the example code did.
Every one of these is caught by review and by standard automated scanning. None requires a specialist. That is the reassuring part.
04What to ask, and what a good answer sounds like
| Ask | A good answer | A worrying answer |
|---|---|---|
| Does a human review everything before it ships? | Yes, by someone other than whoever wrote it, and here is how that is recorded. | "Everything is tested" — which answers a different question. |
| Do you run automated security scanning in the pipeline? | Names a tool, says when it runs, says what happens when it fails. | Yes, without specifics. |
| How do you handle secrets and credentials? | Environment or a secrets manager, never committed, with scanning to catch mistakes. | Any answer containing "in the config file". |
| What proportion of effort is review and testing? | A number, and a note that it went up rather than down as generation got faster. | Surprise at the question. |
| Does any of our code or data go into an AI tool? | Clear yes or no, with the plan tier and training setting named. | "I'd have to check" — acceptable, but get it in writing afterwards. |
| Who can we call if something breaks in eighteen months? | A named arrangement, whether or not it is them. | Nothing specific. This is the most common gap and has nothing to do with AI. |
Our questions, drawn from what we would want a client to ask us. The point of the middle column is that specificity is the signal — a supplier who has genuinely thought about this answers with names and mechanisms, not adjectives.
05What not to do
- Do not ban your supplier from using AI. You will not be able to verify it, you will pay more for the same result, and the good suppliers — who use it with proper review — will decline the work. This is the same dynamic as banning it internally: the practice continues, you just stop hearing about it.
- Do not ask for a "no AI" warranty. Ask for review, scanning and secrets handling instead. Those are verifiable and they are what actually protects you.
- Do not assume a cheaper quote reflects the same process. AI has widened the gap between suppliers who kept their review discipline and those who quietly dropped it, and the price often does not reveal which you are getting.
- Do not skip the boring ask: who holds the code, the hosting and the keys, and what happens if you part ways. Still the most common way a small business gets stuck, and entirely unrelated to AI.
06The honest summary
A meaningful share of AI-generated code contains a security flaw before review — somewhere between a quarter and nearly half depending on whose test you read. Output volume has risen 3–4x while security findings rose around 10x, and developers using these tools have been measured writing less secure code while feeling more confident about it.
None of that is an argument against AI-assisted development, and we would not make one, because we use it. It is an argument that the review step became more important at exactly the moment it became harder to keep up with.
So do not ask whether your developer uses AI. Ask who reviews it, what scans it, where the secrets live, and who you call in eighteen months.
07Common questions
How much AI-generated code has security flaws?
Estimates range from about a quarter to nearly half, depending on the test. Veracode's spring 2026 testing across more than 100 models found 45% of samples introducing an OWASP Top 10 vulnerability; a separate 2026 study of 522 samples across six models found 25.7%. The spread reflects genuinely different models, task types and vulnerability definitions, and crucially whether raw first output or reviewed code was measured. All of them measure output before review, which is what review exists to catch.
Should I ask my developer whether they use AI?
It is the wrong question. The answer is almost certainly yes, it does not distinguish a good supplier from a bad one, and a developer who says no may simply not be telling you. Ask instead who reviews code before it ships, what automated security scanning runs in the pipeline, how secrets and credentials are handled, and what proportion of effort goes into review and testing.
What is the most concerning finding about AI-assisted development?
Not the vulnerability rate — the false confidence. Research has repeatedly found developers using AI assistants writing less secure code while rating their own insecure solutions as secure. That is worse than either half alone, because the usual safety net in software is a competent person feeling uneasy, and code that arrives formatted correctly and confidently explained does not trigger that feeling.
Does AI-assisted development actually save money?
It produces more code faster — research across large enterprises found AI-assisted developers committing at three to four times the rate of peers. But the same research found security findings rising about tenfold. If a supplier's speed increased and the price did not, that time went somewhere: often into more scope for the same money, sometimes out of testing and review. Asking what proportion of effort is review and testing is a fair way to find out which.
What kinds of bugs does AI-generated code produce?
Ordinary ones. Input handling that assumes well-formed data, producing injection flaws. Endpoints that do what was asked without checking the caller was allowed, which is how one customer reads another's records. Secrets committed into code or config because the fastest working example put them there. Outdated or occasionally non-existent dependencies. And error handling that surfaces stack traces to users. All are caught by ordinary review and standard automated scanning.
Should I ban my software supplier from using AI?
No. You cannot verify it, you will pay more for the same result, and suppliers who use it with proper review will decline the work — leaving you with those who either do not use it or do not admit it. It is the same dynamic as banning AI internally: the practice continues and you stop hearing about it. Ask for review, scanning and secrets handling, which are verifiable and are what actually protects you.
Does our code or data go into an AI tool when a developer uses one?
It can, depending on the tool, the plan tier and the training setting. This is a fair question to ask any supplier, and a good answer names the tool and the setting rather than offering reassurance in general terms. If a supplier needs to check, that is reasonable — get the answer in writing afterwards, particularly if your codebase contains customer data or credentials.
What should I ask that has nothing to do with AI?
Who holds the code, the hosting and the keys, and what happens if you and the supplier part ways. This remains the most common way a small business gets stuck with software it cannot maintain or move, and it long predates AI. A build you cannot host, modify or hand to another developer is a dependency rather than an asset, however well it works on the day it ships.
Send these six questions to your developer
Take the questions in section 04, send them to whoever builds or maintains your software, and forward us the replies. We will tell you which answers are solid, which are vague in a way that matters, and which are fine. If your supplier comes back well, we will say so — that is a useful thing to know about someone you depend on.
Ask for a process checkSources, read 9 September 2026: Veracode's spring 2026 GenAI code security testing across more than 100 models for the 45% OWASP Top 10 figure; a separate 2026 analysis of 522 samples across six models for the 25.7% figure; published research across large enterprises for the 3–4x commit rate against roughly 10x security findings; and academic work finding AI-assisted developers producing less secure code while rating it secure. Several of these are vendor studies, and all measure model output under test conditions rather than any particular developer's finished work — they describe raw output before review, which is the case for review rather than against the tools. The question table and failure list are our own. Related: Custom AI Software for Trades and Your Staff Are Already Using AI You Didn't Approve.
Hero image from Unsplash, used under the Unsplash License.