Facial emotion recognition performs well on posed, frontal, well-lit expressions and considerably worse on spontaneous ones. Reported engagement classification accuracy of around 91.5% from facial data alone rises to 94.6% when behavioural signals are added. The underlying assumption — that discrete emotions map reliably to facial configurations — is scientifically contested.
Emotuit has a facial analysis configuration, so we have a commercial interest in you believing this technology works. We are publishing the limitations anyway, because the alternative is you discovering them after purchase — and because a vendor who only publishes strengths should be read accordingly. That includes us on every other page.
The headline numbers
Two things are worth noticing about those numbers.
First, 91.5% sounds excellent — until you see that adding non-biometric behavioural signals gets you to 94.6%. The facial layer contributes roughly three percentage points on top of data you can collect from browser events.
Second, and more importantly: accuracy against what? These figures measure agreement with a labelled dataset. The label is usually either a posed expression or a human annotator's judgement. Neither is the same as the person's actual internal state, which is unobservable. Every accuracy figure in this field is really a measure of agreement with a proxy.
Posed versus spontaneous
This is the single largest gap between marketing claims and field performance.
Most benchmark datasets consist of posed expressions — participants instructed to display anger, surprise, disgust. Posed expressions are exaggerated, prototypical, frontal and well-lit. Classifiers do very well on them.
Spontaneous expressions, produced by someone genuinely reacting to a training module, are quite different: subtler, briefer, frequently asymmetric, often blended, and regularly absent entirely. A person can be deeply frustrated and display almost nothing.
People watching content passively often display very little facial activity at all. Not because they feel nothing, but because there is no social partner to signal to. Facial expression is substantially communicative — it is something we do at other people. Someone alone at a desk has limited reason to produce one, which is precisely the scenario e-learning analytics is deployed in.
This is why baseline calibration and change-over-time measurement matter more than absolute classification. Tracking that a person's affect shifted during a segment is far more defensible than asserting what they felt.
The contested science
The technology rests on an assumption from Ekman's Facial Action Coding System: that a small set of basic emotions map to recognisable, largely universal facial configurations.
That assumption has been seriously challenged. A major review led by Lisa Feldman Barrett concluded that the mapping between facial configurations and emotional states is far weaker and more context-dependent than the basic-emotion model implies — people scowl when angry considerably less often than the model predicts, and scowl for many reasons other than anger.
The practical implication is not that the technology is useless. It is that the inference is probabilistic and context-dependent, and confident individual-level claims are not supported.
Regulators have noticed. The ICO has publicly warned about biometric technologies claiming to detect emotional states, citing the weak scientific footing and the discrimination risk when such systems do not work as claimed.
Demographic variance
Performance is not uniform across populations. Documented sources of variance include:
- Training data composition. Models trained on demographically narrow datasets perform worse on under-represented groups. This is a well-documented pattern across computer vision.
- Cultural display rules. Norms about which emotions may be shown, to whom, and how intensely differ substantially between cultures. A classifier trained predominantly on one population encodes that population's norms as the baseline.
- Neurodivergence. Autistic people may produce facial expressions that differ systematically from neurotypical patterns. A system trained on neurotypical faces will misread them — and this has been a substantive objection in the proctoring debate specifically.
- Age. Facial morphology changes across the lifespan; models trained mostly on adults degrade on children and older adults.
Per-user baseline calibration mitigates some of this, because it measures deviation from that individual's own neutral rather than a population norm. It does not eliminate it.
Physical failure modes
More mundane, and in practice they cause most day-to-day error:
| Condition | Effect |
|---|---|
| Poor or backlit lighting | Landmark detection degrades sharply |
| Glasses, especially with glare | Eye-region features obscured |
| Facial hair | Mouth-region action units obscured |
| Off-axis camera angle | Laptop cameras look up; models expect frontal |
| Low-resolution webcam | Subtle action units unresolvable |
| Face partially out of frame | Detection fails intermittently |
| Masks | Lower face entirely unavailable |
None of these are exotic. A learner on a laptop in a dim room wearing glasses is an ordinary case, and it is close to the worst case for this technology.
What it is genuinely good at
Having spent seven sections on limitations, the honest other side:
Relative change within a person. Did this individual's affect shift between segment A and segment B? With a per-user baseline this is reasonably reliable, and it is the measurement most engagement use cases actually need.
Aggregate patterns across many people. Individual error partially cancels at scale. "This segment produced more negative affect across 400 viewers" is far more defensible than any single-person claim.
Detecting presence and gross state changes. Whether someone is there, whether they turned away, whether something changed markedly.
Timing. Even where the emotion label is uncertain, when the affective change occurred is often accurate, and for content diagnostics timing is frequently the whole answer.
What this means for buyers
Ask what the accuracy figure is measured against. Posed dataset or spontaneous field data? The difference is large and the answer is rarely volunteered.
Be sceptical of individual-level claims. Aggregate and relative measurement is defensible. "This learner was frustrated at 14:32" is not.
Never make consequential decisions about individuals from it. Not performance management, not academic integrity, not hiring. The error rate is not low enough and the failure modes are not evenly distributed.
Check whether you can lawfully use it at all. In EU workplace and education contexts, emotion inference from biometric data has been prohibited since February 2025. This question comes before the accuracy question.
Ask what you lose without it. In our case: roughly three percentage points of engagement classification accuracy, and emotional texture — the ability to distinguish confusion from boredom. For most content diagnostics, behavioural signals answer the question without any of the above.
The summary we would give a prospect on a call: facial emotion recognition is a real technology with real uses, oversold by most of its vendors, resting on a contested scientific premise, and — for the specific job of working out which part of your training loses people — largely unnecessary.
Frequently asked questions
Is facial emotion recognition accurate enough to use?
Why do vendors quote such high accuracy figures?
Does per-user baseline calibration fix the accuracy problem?
Should I avoid facial emotion recognition entirely?
- Barrett et al. — Emotional Expressions Reconsidered (Psychological Science in the Public Interest)
- OUCI/DNTB — multimodal engagement classification in online learning
- PMC / Multimedia Tools and Applications (2022) — engagement indices from facial emotion recognition
- ICO — biometric technologies and emotion detection
The configuration that avoids all of this
Signals measures engagement from behavioural telemetry alone — no camera, no emotion inference, no accuracy ceiling to argue about.