Executive Function
Confidence Interval Task
Also called: overprecision task, interval estimation task, 90% confidence interval task, subjective confidence intervals, calibration of interval estimates
Measure overprecision: participants give a best guess and a 90% range for the year of each of 20 events, scored as the share of ranges that include the right year.
8-12 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 71
- Evidence grade
- D
- Registered clinical studies
- 0
What it measures
Measures overprecision, being too sure of what one knows. For each of 20 questions about the year of an event, written for this test in each of the eight languages, participants type a best guess and a range of years they are 90% sure contains the right year. The main score is the share of ranges that contain it.
Constructs
- Overprecision
- Overconfidence
- Calibration
- Metacognition
- Interval estimation
Research uses
- Judgment and decision-making research
- Overconfidence and calibration research
- Forecasting and expert judgment research
- Within-study comparison of confidence levels
How the paradigm works
The Confidence Interval Task measures overprecision, the excessive certainty that one knows the truth and the most persistent of the three kinds of overconfidence (Moore and Healy, 2008). Each of 20 questions, written for this task in each language, asks for the year of an event from 1500 to 1899, such as when a painting was made or a planet was discovered, five from each century. The participant types a best guess, then an earliest and a latest year such that they are 90% sure the right year lies between them; the earliest year cannot follow the best guess and the latest cannot precede it. There is no time limit and no feedback, and the order comes from the session's seed. One practice question comes first and shows its answer. A setting asks for 80% or 50% ranges instead.
More about the paradigm
The main score is the hit rate, the share of ranges that include the right year; a well calibrated person's 90% ranges include it about 90% of the time. Overprecision is the confidence level minus the hit rate. The width ratio compares the median width of the ranges with the width that would have held the right year at that level around the person's own best guesses: below 1, the ranges were narrower than the person's own accuracy called for, the pattern Soll and Klayman (2004) report, with intervals sometimes only 40% as large as needed. The median width, the median error of the best guesses and the shares of ranges that end too early or start too late are reported too. Moore and Healy (2008) note that 90% intervals contain the answer less than half the time in the usual question format. Scores describe ranges given for these questions; they are research measures and do not judge a person's knowledge.
Scores
- Hit Rate%
- Share of ranges that include the right year, ends included.
- Calibrated at the confidence level asked, 0.9 as standard; below it, ranges were too narrow
- Overprecision%
- The confidence level minus the hit rate.
- 0 is calibrated; above 0, ranges were too narrow
- Width Ratioscore
- Median range width over twice the confidence-level quantile of the best guesses' errors: the width that would have held the right year at that level around the person's own best guesses.
- 1 matches the person's own accuracy; below 1, ranges were narrower than it called for
- Median Range Widthyears
- Median of the latest year minus the earliest year.
- Informational
- Median Best-Guess Erroryears
- Median distance of the best guesses from the right years.
- Lower is more accurate
- Ranges Too Early%
- Share of ranges whose latest year is before the right year.
- Informational
- Ranges Too Late%
- Share of ranges whose earliest year is after the right year.
- Informational
- Single-Year Rangescount
- Ranges whose earliest and latest year are the best guess.
- On half the questions or more is flagged
- Confidence Level%
- How sure the participant was asked to be: 0.9 as standard, or 0.8 or 0.5.
- Informational
- Median Answer Timems
- Median time from a question appearing to its three years being submitted, for answers of at least 100 ms.
- Informational; the task has no time limit
- Answers Under 100 mscount
- Answers submitted less than 100 ms after their question appeared.
- 0 expected
- Questions Answeredcount
- Questions with a best guess and a range in the scored run.
- Fewer than half the planned questions invalidates
- Planned Questionscount
- Questions the run planned: 20.
- Informational
Use and evidence
71 papers from 2021 to 2025 name this task in their title or abstract, up 87% compared with 2016 to 2020. That ranks it 363 of 711 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 25 |
| 2024 | 11 |
| 2023 | 10 |
| 2022 | 14 |
| 2021 | 11 |
| 2020 | 9 |
| 2019 | 11 |
| 2018 | 8 |
| 2017 | 6 |
| 2016 | 4 |
| 2015 | 7 |
| 2014 | 4 |
| 2013 | 4 |
| 2012 | 0 |
| 2011 | 3 |
| 2010 | 3 |
| 2009 | 2 |
| 2008 | 1 |
| 2007 | 1 |
| 2006 | 0 |
| 2005 | 0 |
| 2004 | 0 |
| 2003 | 0 |
| 2002 | 0 |
| 2001 | 0 |
| 2000 | 0 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 11
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes finance papers on overconfident investors.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.58.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Questions
- 20 questions about the year of an event from 1500 to 1899, 5 from each century, written for this library in each language: paintings, music, books, science, inventions, buildings and sport. Each year is checked against a named public source. No war, politics, religion or illness.
- Answer
- Three typed years on one screen: a best guess, then an earliest and a latest year that the participant is 90% sure include the right year. The earliest year cannot follow the best guess and the latest cannot precede it.
- Order
- Drawn from the session's seed. One practice question comes first, followed by its answer and whether the range included it; the 20 questions give no feedback and have no time limit.
- Setting
- How sure the ranges should be: 90% (standard), 80% or 50%. Overprecision is scored against the level chosen.
- Scores
- The hit rate, the share of ranges that include the right year (the main score); overprecision, the level minus the hit rate; the width of the ranges against the width the person's own best guesses called for; the median range width and best-guess error in years; and the shares of ranges that end too early or start too late.
The questions are this library's own, and the Second Estimate Task asks 20 others from the same bank, so a study can run both without repeating a question. Hit rates depend on the questions, so compare conditions or groups within a study. The scores describe ranges given for these questions; they support research and do not judge a person's knowledge.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: confidence_intervals_item from the run with the most answered questions, latest on ties. Typed years have no chance level, and narrow ranges that miss the right year are the result of interest, so the hit rate never invalidates. Fewer than half the planned questions answered invalidates, and any other incomplete run is flagged. Ranges of a single year on at least half the questions are flagged for review. Answers under 100 ms are counted as anticipations, and answers under 200 ms on at least half the questions invalidate. A hidden tab withholds the percentile.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. Ranges that are too narrow for the person's knowledge are what the task measures, so a low hit rate is a result, not a failed task. A range that is only the best guess repeated may mean the range was not understood, so it is flagged and never excludes. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. A hidden tab could allow a search for the answers, so the percentile is withheld after one.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Range confidence | 90% (standard) | 90% (standard), 80%, 50% |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Confidence Interval Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Soll JB, Klayman J, 2004; Moore DA, Healy PJ, 2008), with the standard settings (range confidence: 90%). The main score was hit rate (share of ranges that include the right year, ends included).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Soll, J. B., & Klayman, J. (2004). Overconfidence in interval estimates. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(2), 299-314. https://doi.org/10.1037/0278-7393.30.2.299Ranges for numerical questions, such as the year of an event, include the answer far less often than the confidence asked, and are too narrow for the accuracy of one's knowledge.
- Moore, D. A., & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502-517. https://doi.org/10.1037/0033-295X.115.2.502Overprecision as one of three kinds of overconfidence, measured with 90% intervals that include the answer less than half the time.
- Haran, U., Moore, D. A., & Morewedge, C. K. (2010). A simple remedy for overprecision in judgment. Judgment and Decision Making, 5(7), 467-476. https://doi.org/10.1017/S1930297500001637Overprecision as prevalent and hard to debias, measured by eliciting confidence intervals.
- Park, S., & Budescu, D. V. (2015). Aggregating multiple probability intervals to improve calibration. Judgment and Decision Making, 10(2), 130-143. https://doi.org/10.1017/S1930297500003910The range, two-point and three-point formats of interval estimates, the last with a best guess beside the two bounds.
Source papers
- Moore DA, Healy PJ (2008). The trouble with overconfidence. Psychological Review, 115(2):502-517.Paradigm source
- No normative data for this task.Paradigm source, N/A
Questions
How long does the Confidence Interval Task take?
About 8 to 12 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Hit Rate, Overprecision, Width Ratio, Median Range Width, Median Best-Guess Error, Ranges Too Early, Ranges Too Late, Single-Year Ranges, Confidence Level, Median Answer Time, Answers Under 100 ms, Questions Answered and Planned Questions. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Confidence Interval Task used in research?
71 papers from 2021 to 2025 name it in their title or abstract, up 87% compared with 2016 to 2020. That ranks it 363 of 711 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.58.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Confidence Interval Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Range confidence (90% (standard), 80%, 50%). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.