The interesting change from AI in software development is not that code gets written faster. It is that a new category of code now exists: code that looks correct, was produced quickly, and was never reasoned about by anyone.
Reviewing that code is a different job from reviewing a colleague’s. Human mistakes cluster around difficulty. Generated mistakes cluster around plausibility — and plausible errors are the ones review is worst at catching.
How generated code fails differently
| Failure | Human author | Generated |
|---|---|---|
| Syntax errors | Occasional | Almost never |
| Obviously wrong logic | Occasional | Rare |
| Subtly wrong edge case | Common | Very common |
| Invented API or parameter | Rare | Common |
| Right answer, wrong context | Rare | Very common |
| Silently ignored error path | Occasional | Common |
| Outdated pattern | Depends on the person | Common — training has a cutoff |
A human writing a tricky function knows it is tricky and flags it. A model produces the tricky function with exactly the same confidence as the trivial one, and nothing in the output signals which is which.
A review checklist for generated code
- Does every API called actually exist? Check the method names and parameter orders against the real documentation, not against how plausible they look.
- What happens on the error path? Generated code frequently catches an exception and continues, or handles the happy path only.
- What are the boundaries? Empty list, single item, maximum size, null, duplicate. Generated code handles the middle of the range confidently and the ends carelessly.
- Is this pattern current? Deprecated approaches appear regularly because they were common in training data.
- Does it fit this codebase? Correct in isolation, wrong here — a different error-handling convention, a duplicate of a utility that already exists ten lines away.
- Is there a security assumption? Input validation, escaping, permission checks. These are exactly the things that get omitted without any visible gap.
The habit that matters most
Do not accept code you could not have written.
Not “would not have bothered writing” — could not. If you do not understand why a line is there, you cannot maintain it, you cannot debug it at 2am, and you cannot tell whether it is correct. Ask for an explanation, read it, and if it still does not make sense, do not merge it.
This is the whole discipline, and it is the one that erodes under deadline pressure, because accepting the suggestion is fast and understanding it is not.
Where it genuinely helps
- Tests. Generating cases you did not think of is a real strength — and a wrong test fails loudly, which is the cheapest possible failure mode.
- Boilerplate. Config, scaffolding, repetitive mappings. Low risk, obvious when wrong.
- Explaining unfamiliar code. Faster than reading a whole module, as long as you verify the claims that matter.
- Translation. Between formats, between languages, between a description and a regex. Verifiable on a sample in seconds.
- First-pass review. Pointing a model at a diff catches a useful share of obvious problems before a human looks. It does not replace the human.
What it does to a team
Two effects show up consistently once a team adopts these tools properly.
Review becomes the bottleneck. More code arrives, and the constraint moves to the people reading it. Teams that respond by reviewing faster get worse outcomes; teams that respond by making changes smaller get better ones.
Juniors need more support, not less. The tools produce output a junior cannot yet evaluate, which is precisely the situation where a wrong answer does the most damage. The gap between “produced working code” and “understands the system” widens, and someone has to close it deliberately.
Common questions
Does generated code have licensing risk?
It is a live and unsettled area. Most business tiers offer indemnification and filters for verbatim matches. If your product is commercial, read what your provider actually commits to rather than assuming.
Should we ban it instead?
Bans produce quiet use, which is worse — the code arrives anyway, unreviewed and undeclared. A short policy about what may be pasted in, and a review standard for what comes out, works better.
Do tests written by AI count as tests?
Only if you read them. A generated test suite that asserts current behaviour rather than intended behaviour will happily lock in a bug forever.
Where else does this pattern apply?
Everywhere the verification cost is high. That single principle predicted every result in our six-task timing test too.
Leave a Reply