A new sales rep finishes the product course on schedule and scores well on the end-of-module quiz. Two weeks later, a buyer says, "We already use a cheaper tool, and our IT team is wary of switching." The rep recites a feature list. The buyer goes quiet, and the call ends without a next step.
This is an illustrative scenario, not a documented customer case.
Nothing in the training record is wrong. The rep completed every module and passed the quiz. Yet the evidence you collected answers a different question from the one the job asks. This article shows how to compare training evidence with the job task, find what is missing, and decide what to change before you scale a programme.
1. Completion is not job readiness
Three different things often get reported as one:
- Participation: the learner attended or completed the course.
- Recall: the learner can state facts, steps or definitions.
- Performance: the learner can do the job task, in context, without a script.
Completion is a delivery measure. It tells you the training reached the person. It says nothing about the other two. A quiz can confirm recall and some prerequisite knowledge, but it only supports a readiness claim when the job task itself is mostly recall.
The Carnegie Mellon Eberly Center puts the principle plainly: assessments should reveal how well people have learned what you want them to learn, while instruction makes sure they learn it. Its alignment guidance asks three questions of any programme: what should learners know and be able to do, which tasks would show that, and which activities prepare them for those tasks (Eberly Center, alignment of objectives, assessments and activities). In the opening scenario, the objective was "handle buyer concerns" but the assessment asked "which feature does X?" The two never met.
Recalling a fact sits at the bottom of the usual objective ladder. Handling the concern sits much higher: the rep must analyse what worries the buyer and evaluate which response suits that buyer. Those are different task demands, so they need different evidence.
2. Diagnose the missing task
Before changing anything, describe what a capable rep would have done in that call. In this scenario it comes down to three actions:
- Interpret the concern. Is the buyer worried about cost, risk, effort to switch, or internal politics?
- Ask a useful question. For example, "What would your IT team need to see before they felt comfortable?"
- Choose a relevant response. Offer something that fits the answer, rather than reciting everything the product does.
Now set that beside the quiz evidence. The quiz asked for recognition of features. It never asked the rep to interpret an ambiguous statement, never required a question, and never asked them to choose between two plausible responses. The gap is not that the rep failed training. The training evidence never sampled the task.
A simple way to run this comparison is a three-column check:
| Job task action | What the assessment captured | Gap |
|---|---|---|
| Interpret the concern | Nothing | Not assessed |
| Ask a useful question | Nothing | Not assessed |
| Choose a relevant response | Feature recall | Partly assessed (recall only) |
If most of your task actions land in the "not assessed" column, you have found the first problem, and it costs nothing to fix on paper.
3. Check non-learning barriers
Some people can do the task in practice and still fail on the job, because something around them blocks them. Before you redesign the course, rule these out:
- Product or offer fit. If the buyer's concern is legitimate and the product genuinely does not fit, no amount of skill closes that deal.
- Unclear authority. Can the rep offer a trial, a discount or a reference call? If they must escalate everything, hesitation is rational.
- Poor information. Do reps have a current comparison sheet, security answers or pricing rules to hand?
- Incentives. Is the behaviour you trained rewarded, ignored or quietly penalised?
You can tell these apart from a practice gap with a quick test. Give the same rep a realistic case in a safe setting, with the information and authority they would have at work. If they handle it well there but not on live calls, suspect a non-learning barrier. If they struggle in both, you have a practice gap. If they succeed in both but the numbers do not move, look at the offer or the market.
Be honest about what remains. Even when task performance improves, constraints such as pricing, policy or workload may stay in place, and training will not remove them. Name them in your report so nobody expects training to.
4. Redesign practice and feedback
Where the diagnosis points to a practice gap, the redesign has a simple shape: case, response, feedback, new case.
- Case. Write a realistic buyer concern, close to what reps hear. Have a sales lead validate it so it reflects real calls.
- Response. Ask the learner to say what they would do, and to explain why. Require them to cite the evidence in the case that supports their choice, and to name what further information or support they would need.
- Feedback. Respond to the decision, the reasoning and the evidence quality, not just the final answer. Right answers reached for weak reasons deserve a flag.
- New case. Give a similar but different concern. Success on the same case only shows memory of the case.
A coach should review the reasoning, not just the outcome, when the case is ambiguous, when a learner picks a defensible but unusual option, or when a learner is confident and wrong. Those are the moments that rubric scores alone can hide.
A starter rubric you can adapt
Score each criterion 0 to 2. This is a proposed rubric to adapt with your sector specialist, not a validated readiness standard.
| Criterion | What to look for |
|---|---|
| Task execution | The learner takes a relevant action and records the evidence that supports it |
| Reasoning | The learner explains constraints, alternatives and uncertainty |
| Boundaries | The learner uses approved procedures and asks for help when information or authority is missing |
Anchors: 0 = absent or unsupported; 1 = partial, with relevant omissions; 2 = complete and justified against agreed criteria. Define critical errors separately, and never let an overall score hide a critical failure.
Where interactive tools fit
The practice cycle above can be run in several formats. Self-paced study can cover prerequisite knowledge, with a brief recall check to confirm it. Interactive or facilitated case practice then handles the reasoning, and live coaching or asynchronous review handles the ambiguous choices. AhaSlides supports interactive learning, adaptive learning, and moving between live and self-paced delivery, so one case sequence can run in a live session and as follow-up work. Check any specific authoring, feedback or reporting function in a demonstration before you plan around it.
5. Test transfer before you scale
A good result in the training room is a hypothesis about the workplace. Test it before you roll the programme out wider.
- Define the observation. Choose a supervised workplace task that matches the case, for example a call where a buyer raises a new concern. Agree in advance what the observer will record, using the same rubric.
- Pick a follow-up date based on opportunity. The CDC's training evaluation guidance separates learning, meaning whether learners met the objectives, from transfer, meaning whether they can apply it back at work, and describes a delayed evaluation as a way to assess transfer once people have had time to apply what they learned. It says the timing should depend on your resources, the topic and learners' capacity to apply the learning (CDC, evaluating training). For reps, that means waiting until they have had real chances to meet this kind of concern, not simply picking a calendar date.
- Compare baseline and unseen tasks. Measure task performance, evidence quality and suitability of the resulting learning design, using a baseline and a task the learner has not practised. State the group, the denominator and the timing.
- Check for other changes. If pricing, territory or tooling changed at the same time, say so before crediting the training.
Avoid promising a revenue effect from this exercise. It tells you whether the skill transferred, which is one input to results, not a forecast of them.
Evidence that shows learners can do the job task
To answer the question this article started with, you would want to see:
- A task description written in observable actions, agreed with someone who does the job.
- Assessment items or cases that sample those actions, not only the content that taught them.
- Responses to new cases, with reasoning and cited evidence.
- Supervised workplace observation after a suitable delay.
- A note of the barriers that remain, such as authority, information or incentives.
If you can produce those five, you can say something meaningful about readiness. If you can only show completion and a quiz score, say exactly that, and describe it as evidence of delivery and recall.
Limits of this approach
This guide structures the diagnosis; it does not supply sector standards. Have a practitioner who does the job validate each case and answer key, and a qualified specialist approve any safety, clinical or legal content.
Try it with your own programme
Start with the three-column check in section 2 on one live course. If you find unassessed task actions, turn one of them into a case and run it as an interactive session.
To see how that case-practice session could run live or self-paced, explore AhaSlides.
