Attention
Triplet Odd-One-Out Task
Also called: triplet task, triplet similarity task, odd-one-out similarity judgments, triadic comparisons
Measure object similarity judgments: participants choose which of three object pictures is least like the other two, scored against the choice most people are expected to make.
5-7 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 53
- Evidence grade
- C
- Registered clinical studies
- 2
What it measures
Measures object similarity judgments with the triplet odd-one-out method. Participants see three object pictures and choose the one least like the other two. Scores how often choices agree with the choice most people are expected to make, and how consistent repeated choices are.
Constructs
- Object similarity judgment
- Semantic knowledge of objects
- Categorization
- Consistency of judgment
Research uses
- Research on object and concept representations
- Semantic cognition and aging studies
- Comparing human and model similarity judgments
- Within-study group comparison
How the paradigm works
The triplet odd-one-out task measures how a person judges the similarity of everyday objects. Each screen shows three pictures of objects in a row, and the participant clicks or taps the one least like the other two. There is no time limit and no right or wrong answer; the instructions ask for a first impression of the objects themselves, not of how the pictures are drawn. A choice is followed by 500 ms before the next triplet.
More about the paradigm
The method follows Hebart and colleagues (2020, 2023), who collected millions of such choices on photographs of 1,854 objects to map the dimensions people use to compare objects. This task uses a fixed set of 60 triplets of object pictures from Noto Emoji, the same for every participant. In each triplet two pictures share a category the third lacks: in 30 the pair shares a broad kind (two animals and a backpack), and in 30 all three share a broad kind and the pair a narrower one (two mammals and a bird). The picture outside the pair's category is the choice most people are expected to make. Where it sits is balanced across the three places. Twenty of the triplets come back later, at least 16 triplets after their first showing, with every picture in a new place. The standard session is 80 choices after 3 practice triplets, in about 6 minutes; a setting shows fewer triplets again or none.
The main score is agreement: the share of triplets on which the participant chose the expected picture. Consistency is the share of repeated triplets answered with the same picture both times; chance is one in three. Agreement for each kind of triplet and the median choice time are also reported. Agreement describes how typical a person's similarity judgments are, and low agreement on the clearly different triplets can also mean the task was not followed. Scores support research and do not diagnose.
Scores
- Agreement%
- Share of triplets, first showings, on which the chosen picture was the expected majority choice: the one outside the category the other two share.
- How typical the choices are; not better or worse
- Consistency%
- Share of repeated triplets on which the same picture was chosen both times. Chance is 0.33.
- Higher means more consistent judgments
- Agreement, Different Kinds%
- Agreement on the 30 triplets that pair two objects of one broad kind with one of another.
- Low values can mean the task was not followed
- Agreement, Same Kind%
- Agreement on the 30 triplets whose three objects share a broad kind.
- How typical the finer judgments are
- Median Choice Timems
- Median time from the triplet appearing to the choice, over all test choices.
- Informational
- Choices Under 100 mscount
- Choices made less than 100 ms after the triplet appeared.
- Lower is better
- Triplets Answeredcount
- Test triplets answered at their first showing.
- Informational
- Repeats Answeredcount
- Repeated triplets answered.
- Informational
Use and evidence
53 papers from 2021 to 2025 name this task in their title or abstract, up 179% compared with 2016 to 2020. That ranks it 249 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 23 |
| 2024 | 15 |
| 2023 | 7 |
| 2022 | 6 |
| 2021 | 2 |
| 2020 | 3 |
| 2019 | 6 |
| 2018 | 0 |
| 2017 | 5 |
| 2016 | 5 |
| 2015 | 1 |
| 2014 | 1 |
| 2013 | 1 |
| 2012 | 2 |
| 2011 | 1 |
| 2010 | 3 |
| 2009 | 1 |
| 2008 | 4 |
| 2007 | 0 |
| 2006 | 0 |
| 2005 | 2 |
| 2004 | 0 |
| 2003 | 0 |
| 2002 | 0 |
| 2001 | 0 |
| 2000 | 0 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 5
- Papers on reliability or validity
- 13
- Registered clinical studies using it as an outcome
- 2
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Pictures
- Everyday objects and animals from Noto Emoji, three in a row, chosen by a click, a tap or Enter, with no time limit.
- Triplets
- 60 triplets written for this test, the same for everyone: 30 pair two objects of one broad kind with one of another, and 30 keep all three in one kind with two of a narrower kind.
- Expected choice
- The picture outside the category the other two share, in each place on a third of the triplets.
- Repeats
- 20 triplets come back at least 16 triplets after their first showing, every picture in a new place (standard; 0 to 20).
- Timing
- 500 ms between triplets.
- Practice
- 3 triplets, each choice marked for 1,000 ms.
The triplets are written for this test with Noto Emoji pictures; no THINGS image or published triplet set is used. The expected choice is set by the triplet's design, not counted from earlier participants.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: odd_one_out_choice test rows from the scored run, first showings and repeats. A choice agrees when it is the expected majority choice (chance 1/3, the expected picture equally often in each place). Agreement below chance invalidates and not above chance is flagged. One place on 90% of choices invalidates when agreement is not above chance. Choices under 200 ms on at least half the triplets invalidate.
Why these rules
Choosing the picture most like the other two, the instruction reversed, gives agreement below chance. Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Triplets shown again | 20 | 0 to 20 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Triplet Odd-One-Out Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Hebart MN, Zheng CY, Pereira F, Baker CI, 2020; Hebart MN et al., 2023), with the standard settings (triplets shown again: 20). The main score was agreement (share of triplets, first showings, on which the chosen picture was the expected majority choice: the one outside the category the other two share).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Hebart, M. N., Zheng, C. Y., Pereira, F., & Baker, C. I. (2020). Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(11), 1173-1185. https://doi.org/10.1038/s41562-020-00951-3
- Hebart, M. N., Contier, O., Teichmann, L., Rockter, A. H., Zheng, C. Y., Kidder, A., Corriveau, A., Vaziri-Pashkam, M., & Baker, C. I. (2023). THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior. eLife, 12, e82580. https://doi.org/10.7554/eLife.82580The instructions, the 500 ms between triplets and the repeats at least 16 triplets apart.
Source papers
- Hebart MN, Zheng CY, Pereira F, Baker CI (2020). Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4(11):1173-1185.Paradigm source, Adults
- Hebart MN, Contier O, Teichmann L, Rockter AH, Zheng CY, Kidder A, Corriveau A, Vaziri-Pashkam M, Baker CI (2023). THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior. eLife, 12:e82580.Paradigm source, Adults
Questions
How long does the Triplet Odd-One-Out Task take?
About 5 to 7 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Agreement, Consistency, Agreement, Different Kinds, Agreement, Same Kind, Median Choice Time, Choices Under 100 ms, Triplets Answered and Repeats Answered. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Triplet Odd-One-Out Task used in research?
53 papers from 2021 to 2025 name it in their title or abstract, up 179% compared with 2016 to 2020. That ranks it 249 of 395 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Triplet Odd-One-Out Task have?
In the published research, 5 papers report its test-retest reliability and 2 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Triplets shown again (0 to 20). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.