Attention
Perceptual Confidence Task
Also called: metacognition task, perceptual metacognition, dot comparison confidence task, type 2 ROC, metacognitive sensitivity
Measure metacognitive sensitivity with a dot comparison and confidence ratings, holding accuracy near 71% with a staircase.
8-10 minTimed: desktop or laptop with a mouse or trackpad
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 306
- Evidence grade
- C
- Registered clinical studies
- 4
What it measures
Measures how well confidence tracks accuracy (metacognitive sensitivity). Two boxes of dots flash briefly; participants choose the box with more dots and rate how sure they are. A staircase keeps accuracy near 71%.
Constructs
- Metacognition
- Confidence
- Perceptual decision making
Research uses
- Metacognition and confidence research
- Within-study group comparison
- Individual differences in insight into performance
How the paradigm works
The perceptual confidence task measures metacognitive sensitivity: how well a person's confidence tells their correct choices from their errors. On each trial two boxes of white dots flash on either side of a cross. Participants press the left or right arrow key for the box with more dots, then rate how sure they are from 1 (guessing) to 6 (certainly correct).
More about the paradigm
The display and staircase follow Rouault et al. (2018, Experiment 2). Each box is a grid of 625 positions; one box always holds 313 dots and the other 313 plus a dot difference. A cross shows for 1000 ms, the dots for 300 ms, and the empty boxes stay until the choice; the chosen box is then highlighted for 500 ms before the rating. Choices and ratings are unspeeded, with a 10 second limit for each. A 2-down 1-up staircase on the log of the dot difference (start 4.2, steps of 0.4, then 0.2, then 0.1) holds accuracy near 71% (Levitt, 1971), so confidence is compared at similar accuracy. The test runs 120 scored trials by default after 12 practice trials with feedback. The dots are shown for a whole number of screen frames at the measured refresh rate, and each trial records its frame timing. Sizes are set in screen pixels without calibration. Every session runs a recorded random order.
The type 2 ROC area is computed from the ratings on correct and error trials as in Fleming and Lau (2014). 0.5 means confidence did not separate correct from incorrect choices; higher values mean it separated them better. Mean confidence, overall and after correct and error choices, accuracy and the mean dot difference are also reported.
Scores
- Type 2 ROC Areascore
- Area under the type 2 ROC curve, from confidence ratings on correct and error choices.
- 0.5 is chance; higher means confidence tracked accuracy better
- Mean Confidencescore
- Mean rating from 1 (guessing) to 6 (certainly correct).
- Higher means more confident overall
- Confidence When Correctscore
- Mean rating after correct choices.
- Higher means more confident when right
- Confidence When Wrongscore
- Mean rating after incorrect choices.
- Lower means less confident when wrong
- Accuracy%
- Share of answered trials on which the box with more dots was chosen.
- The staircase aims at about 0.71
- Mean Dot Differencecount
- Mean difference in dots between the boxes over the scored trials, set by the staircase.
- Lower means smaller differences were judged at the same accuracy
- Ratings Usedcount
- How many of the six ratings were used.
- One rating on every trial is flagged for review
- Median Choice Timems
- Median time from the onset of the dots to the choice.
- Choices are unspeeded
- Dropped Frame Rate%
- Share of display frames the browser dropped while the dots were shown.
- Above 0.05 is flagged for review
Use and evidence
306 papers from 2021 to 2025 name this task in their title or abstract, up 87% compared with 2016 to 2020. That ranks it 82 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 95 |
| 2024 | 58 |
| 2023 | 67 |
| 2022 | 42 |
| 2021 | 44 |
| 2020 | 34 |
| 2019 | 50 |
| 2018 | 33 |
| 2017 | 27 |
| 2016 | 20 |
| 2015 | 25 |
| 2014 | 18 |
| 2013 | 10 |
| 2012 | 10 |
| 2011 | 9 |
| 2010 | 4 |
| 2009 | 4 |
| 2008 | 4 |
| 2007 | 6 |
| 2006 | 0 |
| 2005 | 1 |
| 2004 | 4 |
| 2003 | 0 |
| 2002 | 4 |
| 2001 | 1 |
| 2000 | 1 |
Use grade B
300 to 999 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 7
- Papers on reliability or validity
- 44
- Registered clinical studies using it as an outcome
- 4
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Display
- Two black boxes of white dots, one on each side of a cross; each box is a grid of 625 positions.
- Dots
- One box always holds 313 dots and the other 313 plus the dot difference.
- Sizes
- Screen pixels without calibration, so the visual angle depends on the screen and the viewing distance.
- Timing
- A cross for 1,000 ms, the dots for 300 ms, then empty boxes until the choice; the chosen box is highlighted for 500 ms.
- Keys
- Left and right arrows for the box with more dots, then 1 (guessing) to 6 (certainly correct).
- Pace
- Choices and ratings are unspeeded, with a limit of 10 seconds each.
- Staircase
- 2-down 1-up on the log of the dot difference, starting at 4.2 (67 dots), with steps of 0.4, then 0.2, then 0.1; it aims at about 71% correct.
- Length
- 120 scored trials (60 to 300), with a break after every 40.
- Practice
- 12 trials with feedback; the staircase starts here and carries on.
- Frames
- The dots last a whole number of screen frames at the measured refresh rate; more than 5% dropped frames is flagged for review.
This version follows the display, timing and staircase of Rouault et al. (2018, Experiment 2) and scores the type 2 ROC area of Fleming et al. (2010) as computed by Fleming and Lau (2014). Rouault et al. ran 210 trials and reported meta-d' ratios; the default here is 120 trials.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: perceptual_confidence_test from the scored run, left or right arrow for the box with more dots (chance 50%, the larger box on each side on half the trials), then a rating from 1 to 6. Below chance invalidates and not above chance is flagged. One key on 90% of answers invalidates when accuracy is not above chance. Most choices under 200 ms are flagged. Accuracy outside 60% to 85%, one rating on every trial, too few rated errors or correct choices for the type 2 ROC area, or more than 5% of display frames dropped is flagged for review.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). The staircase holds accuracy near 71% so that confidence can be compared at similar accuracy (Fleming and Lau 2014); accuracy outside 60% to 85% and one rating on every trial were exclusion rules of Rouault et al. (2018), and a display that drops frames changes how long the dots were seen.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Scored trials | 120 | 60 to 300 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Timed test: participants need a desktop or laptop with a mouse or trackpad. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Perceptual Confidence Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Fleming et al., 2010; Fleming and Lau, 2014), with the standard settings (scored trials: 120). The main score was type 2 ROC area (area under the type 2 ROC curve, from confidence ratings on correct and error choices).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010). Relating introspective accuracy to individual differences in brain structure. Science, 329(5998), 1541-1543. https://doi.org/10.1126/science.1191883
- Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8, 443. https://doi.org/10.3389/fnhum.2014.00443Type 2 ROC area and its computation.
- Rouault, M., Seow, T., Gillan, C. M., & Fleming, S. M. (2018). Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance. Biological Psychiatry, 84(6), 443-451. https://doi.org/10.1016/j.biopsych.2017.12.017Dot comparison display, timing and staircase.
- Levitt, H. (1971). Transformed up-down methods in psychoacoustics. Journal of the Acoustical Society of America, 49(2B), 467-477. https://doi.org/10.1121/1.1912375The 2-down 1-up rule converges on about 71% correct.
Source papers
- Fleming SM, Weil RS, Nagy Z, Dolan RJ, Rees G (2010). Relating introspective accuracy to individual differences in brain structure. Science, 329(5998):1541-1543.Paradigm source, Adults
- Fleming SM, Lau HC (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8:443.Paradigm source, Adults
- Rouault M, Seow T, Gillan CM, Fleming SM (2018). Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance. Biological Psychiatry, 84(6):443-451.Paradigm source, Adults
Questions
How long does the Perceptual Confidence Task take?
About 8 to 10 minutes, as listed in the test library.
What device does a participant need?
Timed test: participants need a desktop or laptop with a mouse or trackpad. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Type 2 ROC Area, Mean Confidence, Confidence When Correct, Confidence When Wrong, Accuracy, Mean Dot Difference, Ratings Used, Median Choice Time and Dropped Frame Rate. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Perceptual Confidence Task used in research?
306 papers from 2021 to 2025 name it in their title or abstract, up 87% compared with 2016 to 2020. That ranks it 82 of 395 test families in the ConductCognition test census (use grade B). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Perceptual Confidence Task have?
In the published research, 7 papers report its test-retest reliability and 4 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Which settings can a researcher change?
Scored trials (60 to 300). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.