AI Coding Assistants: Where They Help and How to Review Their Output

AI coding tools are genuinely useful and genuinely capable of producing confident nonsense. Getting value from them is mostly about knowing which of those you are looking at.

Where they are strongly positive#

Boilerplate and scaffolding. A CRUD route handler, a form with validation, a config file. The shape is conventional and errors are obvious.

Tests. Generating cases from an implementation is well-suited: the model enumerates edge cases you might skip, and a failing test is self-verifying.

Explaining unfamiliar code. "What does this regex do?" or "walk me through this reducer" is fast and low-risk — a wrong explanation is usually visibly wrong.

Translation. Between languages, between frameworks, between data formats. The semantics are constrained by the input.

The things you know exist but not the syntax for. A git filter-repo invocation, a complex awk pipeline, a CSS grid template. Faster than searching.

Where they are risky#

Anything security-relevant. Missing authorisation checks are the most common serious defect: generated code frequently authenticates without authorising, or trusts a client-supplied id. Hand-rolled crypto is another recurring one.

Version-sensitive APIs. Training data skews toward older majors. Code for React 18 patterns, Next.js Pages Router, or Zod 3 will appear confidently when you are on the newer version.

Invented APIs. A method that sounds exactly right for the library and has never existed. This is the failure mode that costs the most time, because the code looks correct.

Architecture. A model cannot weigh constraints you have not stated — team size, deadline, existing debt, operational capacity. It will produce a defensible-sounding recommendation with no knowledge of your situation.

Performance claims. Generated "optimisations" are frequently neutral or negative. Measure.

A review checklist for generated code#

Read it as you would a PR from someone competent but unfamiliar with the codebase:

1. Does every API it uses actually exist, at our version? Check anything unfamiliar against real docs.
2. Is input validated at the boundary? Especially anything reaching a database or a shell.
3. Is authorisation checked, not just authentication? "Is this a valid user" and "may this user do this to this record" are different.
4. Are errors handled, or swallowed? An empty catch is a common pattern in generated code.
5. What happens on empty, null, and maximum input? Edge cases are the weakest area.
6. Does it match our conventions? Or has it introduced a second way of doing something we already do.
7. Has it added a dependency? Often unnecessarily.
8. Do the tests actually assert something meaningful? Generated tests sometimes assert that a mock was called rather than that behaviour is correct.

Practices that improve the ratio#

• Small requests. A function is reviewable; a feature is not. The review cost of a large generated change often exceeds the writing cost.
• Supply project context. A committed instructions file describing stack, versions and conventions makes output dramatically more consistent.
• Ask for tests with the code. Then run them.
• Never merge code you do not understand. This is the whole discipline in one line. If you cannot explain why it works, you cannot maintain it, and you certainly cannot debug it at 2am.

The honest productivity picture#

The measured effects vary a lot by task, and self-reported speedups tend to exceed measured ones. The pattern that holds up: large gains on well-specified, conventional, verifiable work; small or negative gains on novel problems in unfamiliar code, where review and correction cost more than writing.

That is a useful heuristic for deciding when to reach for it — and it argues for using these tools most aggressively on exactly the work that is least interesting.

Frequently asked questions#

Will AI replace developers?#

It has substantially changed what the job involves — more specification and review, less typing. The parts it does not touch are deciding what to build, why, and whether the result is correct.

Is generated code a licensing risk?#

There is genuine legal uncertainty about training data and output provenance. Most enterprise offerings include indemnification; check what your plan actually covers before it matters.

Should juniors use these tools?#

Yes, with the caveat that accepting code you cannot explain prevents the learning that makes you able to explain it. Use it to see an approach, then understand it.

How do I keep secrets out?#

Never paste them. Use environment-variable placeholders in any code you share, and check your provider's retention policy for your plan.

Related reading#

• Prompt Engineering for Developers
• LLM Context Windows Explained

References#

• Anthropic: Claude Code documentation
• OWASP Top 10 for LLM Applications