Measuring training effectiveness means tracking four things: whether people engaged with the content, whether they learned, whether their behaviour changed, and whether a business outcome moved. Completion rates address none of these. Start with content-level engagement data, because it is the only layer that tells you which part of your training to fix.
Why completion rates fail
Start with the numbers that describe the problem.
Completion rate has one virtue: every LMS reports it without being asked. That is the entire case for it.
What it cannot tell you:
- Whether anyone was paying attention
- Which section lost them
- Whether anything was retained past Friday
- Whether behaviour changed
- Whether the content or the delivery was the problem
- Whether a shorter version would have worked as well
And there is a worse failure mode than being uninformative. Completion rate can be actively misleading. Mandatory compliance training routinely posts completion above 90% while producing little measurable behaviour change. A metric that reads near-perfect while the underlying activity fails is not a weak signal — it is a broken instrument, and it is the one most L&D dashboards lead with.
The four layers worth measuring
Think of effectiveness as four stacked questions, each harder to measure than the last and each more valuable.
| Layer | Question | When available | Difficulty |
|---|---|---|---|
| 1. Engagement | Did they actually attend to it, and where did they stop? | Immediately | Low |
| 2. Learning | Did knowledge or skill increase? | End of session | Low |
| 3. Behaviour | Are they doing anything differently? | 30–90 days | Medium |
| 4. Outcome | Did a business number move? | 1–2 quarters | High |
Most organisations attempt layer 4 first, because that is what the board asks about, then abandon measurement entirely when the attribution proves impossible.
Invert it. Layer 1 is available on day one, is cheap to collect, and is the only layer that tells you what to change. Layers 3 and 4 tell you whether the programme worked overall — valuable for budget defence, useless for improving next month's module.
Kirkpatrick, and where it stops helping
Donald Kirkpatrick's four levels — Reaction, Learning, Behaviour, Results — have been the vocabulary of training evaluation since 1959. Around 78% of organisations run Level 1 on their programmes; roughly half attempt Level 2. Very few get to Level 4. The Phillips model adds a fifth level converting results to financial ROI.
Where it earns its keep: it gives you and your stakeholders a shared language, and it forces the distinction between "people liked it" and "people changed."
Where it falls down: it was designed for instructor-led classroom training six decades before anyone streamed a module on a phone.
Level 1 in practice means a post-session survey — the "smile sheet". It measures what people remember feeling once it is over, which is a poor proxy for what happened during. Peak-end effects dominate. Someone who checked out for twenty minutes in the middle and enjoyed the last five will rate the session well.
More importantly, Level 1 gives you a score, not a location. "3.8 out of 5" does not tell you that everyone left during the section on incident escalation. For digital content, where you can observe engagement continuously and per section, a survey is a strange thing to rely on.
Use Kirkpatrick as a framework for talking to stakeholders. Do not use it as your data collection design.
Metrics that actually work
Layer 1 — Engagement
| Metric | What it tells you |
|---|---|
| Content-level engagement score | Which section holds attention and which loses it |
| Drop-off point | Exactly where sessions end |
| Focus-adjusted time on task | Real attention, discounting time in a background tab |
| Tab-switch rate | The strongest single predictor of disengagement in the literature |
| Replay and rewind rate | Where people were confused enough to go back |
| Interaction latency | Response time to prompts — a cleaner engagement signal than click counts |
Layer 2 — Learning
Pre- and post-assessment delta, not post-assessment score. Delayed retention checks at 30 days, because immediate recall overstates learning badly. Confidence-weighted scoring, which surfaces confidently-held wrong answers — the most dangerous category in compliance training.
Layer 3 — Behaviour
Manager observation against defined criteria. Task performance data from the systems people actually work in. Error and incident rates in the trained domain. Time-to-competency for new starters.
Layer 4 — Outcome
Trained versus untrained cohort comparison, which is the only credible attribution method available to most organisations. Retention differentials. Productivity per employee. Compliance incident rates.
Content-level engagement data
This is the layer most organisations skip, and it is the one that changes what you do on Monday.
Every other metric evaluates the programme. Content-level engagement data tells you which twelve minutes of a forty-minute module to rewrite.
The distinction matters commercially. "Our compliance training scores 3.8/5" starts a conversation about whether to keep running it. "Module 3 loses 60% of learners between minutes 14 and 19, and the drop is consistent across every cohort" starts a conversation about rewriting minutes 14 to 19. One is an evaluation. The other is a work order.
Collecting it requires instrumenting the content rather than the LMS. The signals are ordinary browser events:
- Page Visibility API — has the learner switched tabs?
- Focus and blur events — has the window lost focus without a tab change?
- Intersection Observer — which section is actually on screen, and for how long?
- Scroll velocity — read, or skimmed?
- Media events — pauses, replays, seeks
- Session end — where they left, and what preceded it
None of this requires a camera, and none of it is biometric data — which matters, because EU AI Act Article 5(1)(f) prohibits inferring emotions from biometric data in workplace and education settings.
The research is encouraging here. A 2024 study identified tab-switching as the single strongest predictor of disengagement in online courses, ahead of both self-regulation and satisfaction measures. The most predictive signal available is also the easiest to collect and the least legally fraught.
Building the measurement stack
A sequence that works, in the order that returns value fastest.
Weeks 1–2 — instrument. Add content-level engagement tracking to one course. One is enough to start; resist the urge to boil the ocean.
Weeks 3–4 — establish a baseline. You cannot say a module underperforms without knowing what normal looks like in your organisation. Industry benchmarks are directionally useful and rarely comparable.
Weeks 5–8 — fix the worst drop-off. Take the single largest drop-off point across your catalogue and rewrite it. Measure again. This is your proof case, and it is worth more than any vendor's case study.
Months 3–4 — add learning delta. Pre/post assessment with a 30-day delayed retention check.
Months 4–6 — add behaviour. Pick one measurable on-the-job indicator. One is fine. A single credible behaviour measure beats five theoretical ones.
Months 6–12 — attempt outcomes. Cohort comparison where you can construct one honestly.
If you do nothing else from this guide: find the single biggest drop-off point in your most important course, and fix it. It is measurable in a fortnight, the improvement is attributable, and it produces the internal credibility that funds everything else.
Reporting to executives
Executives do not want your engagement dashboard. They want three things.
What did we change, and what happened? "We rewrote the escalation section of the compliance module after finding 60% drop-off. Completion of that section rose from 41% to 78%, and assessment scores on those questions improved 22 points." Specific, attributable, obviously a result of a decision you made.
What is it costing us when it fails? Translate drop-off into money. If 40% of a 3,000-person workforce disengages from a mandatory module, you have paid for 3,000 licences and 3,000 hours to deliver content to 1,800 people. That framing lands.
What are we doing next? A prioritised list of the worst-performing content, with expected impact.
What to avoid: completion rate as a headline, satisfaction scores presented as effectiveness, and any ROI figure with an implied precision you cannot defend. A finance director who catches one inflated number will discount every number you present afterwards.
Common mistakes
Measuring the programme instead of the content. Programme-level metrics tell you whether to continue. Content-level metrics tell you what to fix. Only one of those is actionable this month.
Treating surveys as engagement data. They measure recalled feeling, dominated by peak-end effects. Useful for perception, not for diagnosis.
Waiting for perfect attribution. You will never cleanly attribute a revenue change to a training module. Use cohort comparison, state your assumptions, and stop trying to prove causation you cannot establish.
Ignoring the denominator. A 90% completion rate on a module nobody was paying attention to is worse than 60% completion with genuine engagement, because the first one is invisible.
Collecting data you will not act on. If you are not going to rewrite content based on the drop-off data, do not collect drop-off data. Measurement that changes nothing is overhead with a dashboard.
The summary, if you want one line: stop asking whether people finished, and start asking where they stopped. The second question has an answer you can act on, and it is available today.
Frequently asked questions
What is the best single metric for training effectiveness?
What is a good completion rate for corporate e-learning?
How do you measure training effectiveness without surveys?
How long before you can measure training impact?
Is the Kirkpatrick model still relevant in 2026?
See where your content loses people
Book a walkthrough and we will show you the engagement data on your own content.