A developer asks an assistant for a function that checks whether a user may download a report. The code compiles. The happy-path test passes. The pull request looks tidy. Two weeks later a colleague notices that the function checks that the user is signed in, but never checks that the report belongs to them.
This is an illustrative scenario, not a documented incident. It describes the gap that AI-generated code review training has to close: generated code that passes a basic check can still contain an authorisation or concurrency flaw, and the person reviewing it needs to find it and say why it matters.
This guide is for engineering enablement leads and engineering managers. It covers how to describe the performance problem, diagnose the gaps, design preparation and practice, work through a case and assess whether the skill transfers to real reviews. It is about training design. Every defect, test and review rule in your programme should be validated by senior engineers and your security team.
What is AI-generated code review training?
AI-generated code review training teaches developers to prompt an assistant clearly, then evaluate and audit what comes back: is it correct, is it safe, does it behave under concurrent load, is it efficient enough? It is not a course on a particular tool. It trains a review habit that stays the same when the tool changes.
Faktenüberblick
| Vraag | Kort antwoord |
|---|---|
| Wat is de taak? | Audit generated code and substantiate each risk with a test. |
| Wat is je score? | Verification quality and whether the relevant defects were found. |
| Is a quiz on secure coding rules enough? | No. It confirms prerequisites, but not whether a developer can spot a flaw in code that looks plausible. |
| Wie valideert de inhoud? | An engineering manager and your security team. |
| Wat kan training niet oplossen? | Review time, tooling, unclear ownership and pressure to merge quickly. |
| Hoe weet je dat het gelukt is? | A new, unseen code exercise, scored against agreed criteria, with follow-up on real reviews. |
1. Beschrijf het prestatieprobleem
Start with the task: audit generated code and substantiate risks with tests. Auditing is reading with suspicion. Substantiating is turning a suspicion into evidence, usually a test that fails or a reproduction that shows the flaw.
The research suggests why this needs deliberate practice. In a study of GitHub Copilot, Pearce and colleagues generated 1,689 programs across 89 scenarios chosen for known weakness types and found that about 40 per cent of them were vulnerable (Pearce and colleagues, 2021). That was an early tool and a specific set of scenarios, so do not treat the figure as a rate for today's assistants. A separate user study asked participants to write code with and without an AI assistant. Those with the assistant wrote significantly less secure code, and were more likely to believe their code was secure (Perry and colleagues, 2023). The same study found that participants who trusted the assistant less and engaged more with their prompts produced fewer vulnerabilities.
Two lessons follow for training design. Plausible output needs an active reviewer. And confidence is not evidence, so the exercise must ask for the test, not the opinion.
Write the performance statement in one sentence and have an engineering manager check it: "Given a generated change, the developer lists the security, concurrency and efficiency risks they see, and for each one describes or writes a test that would show it."
2. Stel de leer- en contexthiaten vast.
Voordat je iets bouwt, vergelijk de taak met wat er in je huidige programma wordt geoefend en beoordeeld. Drie vragen zijn meestal voldoende om een trainingsachterstand van iets anders te onderscheiden:
- Wordt in de huidige praktijk gebruikgemaakt van de daadwerkelijke taak? If training covers prompt writing but never asks developers to review the output, nobody has practised auditing. If assessment is a multiple-choice test on secure coding rules, it measures recall.
- Wat is de cognitieve belasting? In Bloom's revised taxonomy, judging whether code is safe and choosing the test that proves it sit at the analyse and evaluate levels, and designing a better prompt or a fix reaches the create level. Matching activities to those levels is covered in our guide to Leeractiviteiten afstemmen op de taxonomie van Bloom..
- Is er iets anders dan vaardigheid dat in de weg staat? Check review time, tooling and ownership. If reviewers have five minutes per pull request, no training will produce an audit. If there is no static analysis or test harness in the pipeline, a developer can find a risk but not substantiate it. Report non-training causes to the people who can fix them.
3. Ontwerpvoorbereiding en -oefening
Zodra je weet welke lacunes door training kunnen worden opgevuld, geef je elke fase een specifieke taak.
Voorbereiding. Check prerequisites with a short diagnostic: the common weakness categories your stack faces, how your tests are structured, the review checklist your team uses. Self-paced study suits this stage. A recall check confirms prerequisites only.
Praktijk. Use audit exercises built from realistic generated code. Seed each with a small number of defects of different kinds, such as a missing ownership check, a race between two requests and an inefficient loop, plus some code that is fine. The developer reviews, names each risk, says how serious it is and proposes the test that would show it. Ask for reasoning, not a list of line numbers.
Feedback. Link it to decisions rather than a total score. "You flagged the loop but not the race on the shared counter, and your test would not have caught it because it runs requests one at a time" is more useful than "2 of 3 found".
Live input of asynchrone beoordeling? Live input adds most value when a risk is ambiguous and developers learn from hearing a senior engineer explain why they would block the merge. Asynchronous review is enough for comparing a developer's findings with an annotated expert review. Many teams use both: self-paced preparation first, then a facilitated session on the hard cases.
4. Werk een realistisch voorbeeld uit.
Ter illustratie. Een engineeringmanager dient de casus en de antwoordcriteria te valideren alvorens deze te gebruiken.
De zaak. A generated endpoint returns a customer's invoice by invoice ID. It checks that the caller has a valid session, uses a parameterised query and passes the basic test. The same function increments a shared download counter without a lock.
De taak. The developer states each risk they see, ranks them and describes the test that would substantiate each.
Een eerste poging. "Looks fine. The query is parameterised and the test passes." The developer has treated the passing test as proof and has not asked what the test does not cover.
Feedback en opnieuw proberen. The reviewer asks the developer to name what the session check proves and what it does not. The retry asks them to consider two different customers and two simultaneous requests.
Een serieuzere poging. "The session check proves who is calling, not that the invoice is theirs. I would write a test where customer A requests customer B's invoice ID and expect a refusal. The counter is incremented without synchronisation, so two requests at once could lose an update. I would run a concurrent test and compare the count. The query itself is parameterised, so I would not spend time there." Notice that this answer ranks the risks and ties each to a test.
Waarom elk antwoord verdedigbaar of onvolledig is. Judge each against approved criteria, not against a reviewer's preference. The first attempt follows a habit but omits the evidence. The stronger attempt is specific, prioritised and testable.
5. Beoordeel de capaciteit en plan de overdracht.
Evidence of capability is performance on a new exercise the learner has not seen. Score verification quality and the identification of relevant defects. A simple rubric uses a 0 to 2 scale per criterion:
| Criterium | Observeerbaar bewijs |
|---|---|
| Taakuitvoering | Audits the generated code and records the evidence behind each finding. |
| Redenering | Explains the constraints, alternatives and what remains uncertain, and ranks the risks. |
| Grenzen | Follows the team's review rules and escalates to a security specialist when the risk is beyond their remit. |
Anchors: 0 is absent or unsupported, 1 is partial with relevant omissions, and 2 is complete and justified against the agreed criteria. Define critical errors separately, such as approving code with an unprotected access path, so an overall score cannot hide one. This rubric is a proposal, not a validated readiness standard: adapt it with an engineering manager and pilot it. Our guide to Interactieve oefeningen ontwerpen met nuttige feedback Dit laat zien hoe je feedback schrijft die aansluit op beslissingen.
Plan transfer deliberately. After the exercise, pair each developer with a real pull request that includes generated code and review their findings together, or run a short retrospective on a recent review that missed something.
Waar interactieve levering helpt
Interactive formats help where everyone should commit before they see the group. In a live session, a facilitator can show a generated snippet, ask each developer to type the risks they see and the test they would write, and then reveal the spread before the discussion. The same exercise can run self-paced for preparation. AhaSlides supports interactive learning and moving between live and self-paced delivery. It is a presentation and learning tool, not a place to run code, so run the code exercises in your own environment. Check any specific authoring, feedback or reporting function you plan to rely on against your own workflow. For guidance on choosing formats, see our article on Wanneer moet je adaptief, interactief en live leren inzetten?.
Meet wat er veranderd is
Decide in advance what would show that developers can do the task. Compare verification quality and defect identification on an unseen exercise with a baseline. Define who is included, the denominator and the timing. If results improve, check what else changed in the same period, such as a new static analysis rule, a revised review checklist or a change of assistant, before crediting the training.
Mogelijke beperkingen die kunnen blijven bestaan
- Tijd voor een evaluatie. An audit takes longer than a skim, and deadlines push toward the skim.
- Gereedschap. Without a test harness or scanning in the pipeline, a risk is hard to substantiate.
- Gereedschap wisselen. Assistants change quickly, so keep exercises about the review habit, not one product.
- Dekking van de zaak. One exercise shows a few defect types. A score does not describe a developer's judgement in general.
- Transfer. Performance in a workshop is not the same as performance on a busy review queue.
Name these to sponsors early and plan separate fixes. Training improves what a developer can find and prove. It does not create the time or the tooling.
Veelgestelde vragen
What is AI-generated code review training?
It teaches developers to prompt, evaluate and audit code produced by an AI assistant, finding security, concurrency and efficiency problems and backing each with a test.
Is generated code less secure than human code?
Studies point to risk, but results depend on the tool, the task and the study design. One user study found participants with an assistant wrote less secure code and felt more confident about it, which is a reason to train the review habit.
How do you assess whether developers can audit generated code?
Give a new code exercise and score verification quality and defect identification against agreed criteria. A quiz on secure coding rules confirms prerequisites only.
Do you need a code sandbox for the training?
You need somewhere to run the exercises, which is usually your own environment. Check that any tool you use supports what you need before you rely on it.
Waar te beginnen
Pick one recurring kind of generated change in your codebase. Write the performance statement, have an engineering manager validate the seeded defects and the tests that would show them, and draft a three-criterion rubric with a 0 to 2 scale and a list of critical errors. Pilot it with a small group before you roll it out.
To see how an audit exercise could run live with your own team, and be reused self-paced afterwards, explore AhaSlides.
