Social Cognition
Emoji Emotion Recognition
Also called: facial emotion recognition task, emotion labeling task, facial affect recognition
Measure how accurately people name the emotion a face shows: one emoji face at a time from a fixed, freely licensed set, labeled happy, sad, angry, afraid, surprised, disgusted or neutral, scored by percent correct and the unbiased hit rate for each emotion.
4-6 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 1,197
- Evidence grade
- B
- Registered clinical studies
- 187
What it measures
Measures how accurately people name the emotion a face shows. The faces are emoji from one freely licensed set, shown one at a time, and participants choose happy, sad, angry, afraid, surprised, disgusted or neutral.
Constructs
- Emotion recognition
- Facial expression perception
- Social cognition
Research uses
- Emotion perception research
- Social cognition studies in clinical and nonclinical groups
- Within-study group comparison
How the paradigm works
Emoji Emotion Recognition measures how accurately people name the emotion a face shows. The faces are emoji: drawn faces shown in place of photographs of people. Each trial shows one emoji in the middle of the screen until the participant chooses one of seven words: happy, sad, angry, afraid, surprised, disgusted or neutral. The six emotions are the basic emotions of Ekman (1992) and of computerized emotion recognition tasks such as that of Montagne and colleagues (2007), with neutral added. The words are buttons in the same order on every trial, and the keys 1 to 7 also answer. There is no time limit. Emoji have stood in for faces before: an emoji version of a faces pain scale agreed closely with a numeric rating scale (He and colleagues, 2022).
More about the paradigm
The 19 test emoji come from Noto Emoji, a freely licensed set, and every session shows the same image files. The device's own emoji font is never used, because each platform draws emoji differently and people read them differently (Franco and Fugate, 2020). The emoji were chosen by a stated rule from their Unicode names, their keywords in the Unicode CLDR data and published emoji emotion ratings: four each for happy and sad, three for surprised and two each for angry, afraid, disgusted and neutral, as many as pass the rule. The two disgusted emoji show nausea. Three practice emoji outside the test set come first, each followed by the right answer; the test gives no feedback. As standard each emotion appears on 10 trials, 70 in all, spread evenly over its emoji, in an order drawn from the session's seed with no emotion more than twice in a row and no emoji twice in a row. Researchers can set 8 to 12 trials per emotion.
The primary score is percent correct; chance is 1 in 7 (14.3%). Percent correct and the unbiased hit rate of Wagner (1993) are also reported for each emotion. The unbiased hit rate allows for how often each word is used: the hits squared, divided by the trials of that emotion times the number of times its word was chosen. The confusion matrix counts the words chosen for each emotion shown, and the median correct response time is reported for context. Fischer and Herbert (2021) found that the intended emotion was recognized best in emoji, then in human faces, then in emoticons. Scores support research and do not diagnose.
Scores
- Percent Correct%
- Percent of answered test trials labeled with the emotion the emoji shows. Chance is 14.3% (1 in 7).
- Higher is better
- Mean Unbiased Hit Rate%
- Mean of the unbiased hit rates of the emotions shown.
- Higher is better
- Percent Correct, Happy%
- Percent of happy trials labeled happy.
- Higher is better
- Percent Correct, Sad%
- Percent of sad trials labeled sad.
- Higher is better
- Percent Correct, Angry%
- Percent of angry trials labeled angry.
- Higher is better
- Percent Correct, Afraid%
- Percent of afraid trials labeled afraid.
- Higher is better
- Percent Correct, Surprised%
- Percent of surprised trials labeled surprised.
- Higher is better
- Percent Correct, Disgusted%
- Percent of disgusted trials labeled disgusted.
- Higher is better
- Percent Correct, Neutral%
- Percent of neutral trials labeled neutral.
- Higher is better
- Unbiased Hit Rate, Happy%
- Wagner's unbiased hit rate for happy: hits squared divided by (happy trials x times happy was chosen). 0 when no happy trial was labeled happy.
- Higher is better
- Unbiased Hit Rate, Sad%
- Wagner's unbiased hit rate for sad, computed the same way.
- Higher is better
- Unbiased Hit Rate, Angry%
- Wagner's unbiased hit rate for angry, computed the same way.
- Higher is better
- Unbiased Hit Rate, Afraid%
- Wagner's unbiased hit rate for afraid, computed the same way.
- Higher is better
- Unbiased Hit Rate, Surprised%
- Wagner's unbiased hit rate for surprised, computed the same way.
- Higher is better
- Unbiased Hit Rate, Disgusted%
- Wagner's unbiased hit rate for disgusted, computed the same way.
- Higher is better
- Unbiased Hit Rate, Neutral%
- Wagner's unbiased hit rate for neutral, computed the same way.
- Higher is better
- Confusion Matrixcount
- Answers for each emotion shown and word chosen, keyed shown:chosen, for example afraid:surprised.
- Informational
- Median Correct RTms
- Median time from the face to a correct answer, over answers at or after 100 ms.
- Context for the scores
- Anticipationscount
- Answers under 100 ms. They still count for accuracy.
- Context for the scores
- Longest Run of One Wordtrials
- Most consecutive answers with the same word.
- 8 or more is flagged for review
Use and evidence
1,197 papers from 2021 to 2025 name this task in their title or abstract, up 47% compared with 2016 to 2020. That ranks it 20 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 323 |
| 2024 | 275 |
| 2023 | 221 |
| 2022 | 203 |
| 2021 | 175 |
| 2020 | 177 |
| 2019 | 178 |
| 2018 | 165 |
| 2017 | 145 |
| 2016 | 150 |
| 2015 | 130 |
| 2014 | 122 |
| 2013 | 101 |
| 2012 | 98 |
| 2011 | 92 |
| 2010 | 68 |
| 2009 | 50 |
| 2008 | 50 |
| 2007 | 41 |
| 2006 | 28 |
| 2005 | 23 |
| 2004 | 16 |
| 2003 | 13 |
| 2002 | 15 |
| 2001 | 15 |
| 2000 | 11 |
Use grade A
1,000 or more papers from 2021 to 2025.
Evidence grade B
At least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers.
- Papers on test-retest reliability
- 21
- Papers on reliability or validity
- 266
- Registered clinical studies using it as an outcome
- 187
Note on the counts: includes computer-vision papers.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Faces
- 19 emoji from Noto Emoji, a freely licensed set, shown as the same image files on every device: 4 each for happy and sad, 3 for surprised and 2 each for angry, afraid, disgusted and neutral. The device's emoji font is never used.
- Display
- One emoji in the middle of the screen, up to 200 px wide, until an answer. No time limit.
- Answers
- Seven buttons in a fixed order: happy, sad, angry, afraid, surprised, disgusted and neutral. The keys 1 to 7 also answer.
- Trials
- Each emotion 10 times (standard; 8 to 12), spread evenly over its emoji: 70 trials.
- Order
- Drawn from the session's seed, with no emotion more than twice in a row and no emoji twice in a row.
- Between faces
- A 500 ms blank screen.
- Practice
- 3 emoji outside the test set, each followed by the right answer for 1,500 ms. The test gives no feedback.
- Selection rule
- Face emoji whose Unicode name and CLDR keywords name the emotion, checked against published emoji emotion ratings where they exist. The two disgusted emoji show nausea.
Published emotion recognition tasks show photographs of posed faces. This version shows emoji in their place, each at one intensity, so its scores are not comparable with norms from photographed faces.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: emoji_emotion_recognition_test from the scored run, one of seven labels for each emoji (chance 1 in 7, every emotion shown equally often), with no time limit. Below chance invalidates and not above chance is flagged. One label on 90% of answers invalidates when accuracy is not above chance. Answers under 200 ms on at least half the trials invalidate. Fewer than 28 recorded test trials (four per emotion) invalidate; fewer than planned withhold the percentile. The same label on 8 or more trials in a row is flagged for review.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. The order never shows one emotion on more than two trials in a row, so eight answers in a row with one label include at least two errors and suggest answering without looking at the face. The scores for each emotion rest on that emotion's trials, so a session with fewer than four per emotion is invalid, and the norms come from complete sessions, so a session with fewer trials than planned gets no percentile.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Faces per emotion | 10 | 8 to 12 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Emoji Emotion Recognition task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Ekman, 1992; Wagner, 1993; Montagne et al., 2007), with the standard settings (faces per emotion: 10). The main score was percent correct (percent of answered test trials labeled with the emotion the emoji shows).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3-4), 169-200. https://doi.org/10.1080/02699939208411068
- Wagner, H. L. (1993). On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior, 17(1), 3-28. https://doi.org/10.1007/BF00987006The unbiased hit rate.
- Montagne, B., Kessels, R. P. C., De Haan, E. H. F., & Perrett, D. I. (2007). The Emotion Recognition Task: A paradigm to measure the perception of facial emotional expressions at different intensities. Perceptual and Motor Skills, 104(2), 589-598. https://doi.org/10.2466/pms.104.2.589-598A computerized task with the six basic emotions.
- He, S., Renne, A., Argandykov, D., Convissar, D., & Lee, J. (2022). Comparison of an emoji-based visual analog scale with a numeric rating scale for pain assessment. JAMA, 328(2), 208. https://doi.org/10.1001/jama.2022.7489Emoji in place of a faces scale.
- Franco, C. L., & Fugate, J. M. B. (2020). Emoji face renderings: Exploring the role emoji platform differences have on emotional interpretation. Journal of Nonverbal Behavior, 44(2), 301-328. https://doi.org/10.1007/s10919-019-00330-1Why every session shows the same emoji images.
- Fugate, J. M. B., & Franco, C. L. (2021). Implications for emotion: Using anatomically based facial coding to compare emoji faces across platforms. Frontiers in Psychology, 12, 605928. https://doi.org/10.3389/fpsyg.2021.605928Emoji faces coded against human facial expressions.
- Fischer, B., & Herbert, C. (2021). Emoji as affective symbols: Affective judgments of emoji, emoticons, and human faces varying in emotional content. Frontiers in Psychology, 12, 645173. https://doi.org/10.3389/fpsyg.2021.645173Emoji compared with human faces.
- Shoeb, A. A. M., & de Melo, G. (2020). EmoTag1200: Understanding the association between emojis and emotions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (pp. 8957-8967). https://doi.org/10.18653/v1/2020.emnlp-main.720Emoji emotion ratings used in the selection rule.
- Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1-68. https://doi.org/10.1177/1529100619832930Limits of tasks built on posed expressions.
Source papers
- Ekman P (1992). An argument for basic emotions. Cognition and Emotion, 6(3-4):169-200.Paradigm source, Adults
- Wagner HL (1993). On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior, 17(1):3-28.Paradigm source, Adults
- Montagne B, Kessels RPC, De Haan EHF, Perrett DI (2007). The Emotion Recognition Task: a paradigm to measure the perception of facial emotional expressions at different intensities. Perceptual and Motor Skills, 104(2):589-598.Paradigm source, Adults
- He S, Renne A, Argandykov D, Convissar D, Lee J (2022). Comparison of an emoji-based visual analog scale with a numeric rating scale for pain assessment. JAMA, 328(2):208.Paradigm source, Adults
- Franco CL, Fugate JMB (2020). Emoji face renderings: exploring the role emoji platform differences have on emotional interpretation. Journal of Nonverbal Behavior, 44(2):301-328.Paradigm source, Adults
Questions
How long does the Emoji Emotion Recognition take?
About 4 to 6 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Percent Correct, Mean Unbiased Hit Rate, Percent Correct, Happy, Percent Correct, Sad, Percent Correct, Angry, Percent Correct, Afraid, Percent Correct, Surprised, Percent Correct, Disgusted, Percent Correct, Neutral, Unbiased Hit Rate, Happy, Unbiased Hit Rate, Sad, Unbiased Hit Rate, Angry, Unbiased Hit Rate, Afraid, Unbiased Hit Rate, Surprised, Unbiased Hit Rate, Disgusted, Unbiased Hit Rate, Neutral, Confusion Matrix, Median Correct RT, Anticipations and Longest Run of One Word. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Emoji Emotion Recognition used in research?
1,197 papers from 2021 to 2025 name it in their title or abstract, up 47% compared with 2016 to 2020. That ranks it 20 of 395 test families in the ConductCognition test census (use grade A). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Emoji Emotion Recognition have?
In the published research, 21 papers report its test-retest reliability and 187 registered clinical studies use it as an outcome (evidence grade B in the ConductCognition test census).
Which settings can a researcher change?
Faces per emotion (8 to 12). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.