Reasoning
Hindsight Bias Task
Also called: knew-it-all-along effect, hindsight effect, memory design, recall of original judgments, creeping determinism
Measure how far knowing a true number moves the memory of an earlier estimate: participants estimate pictures of many copies of an object, then recall each estimate, half after seeing the true number, scored as the hindsight bias index.
6-8 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 280
- Evidence grade
- C
- Registered clinical studies
- 4
What it measures
Measures how far knowing the true number moves the memory of an earlier estimate. Pictures of many copies of one object are shown for 2 seconds and the participant estimates how many there were; later each object comes back, half with the true number, and the participant recalls the estimate. The main score is the hindsight bias index: how much closer recalls move toward the true numbers that were shown than toward those that were not.
Constructs
- Hindsight bias
- Memory for judgments
- Knowledge updating
- Numerical estimation
Research uses
- Judgment and decision making research
- Memory distortion research
- Cognitive bias research
- Within-study group comparison
How the paradigm works
The Hindsight Bias Task measures how far knowing a true number moves the memory of an earlier estimate. Fischhoff (1975) showed that an outcome people know seems more likely than it did beforehand, even when they try to set the knowledge aside. In the memory design (Fischhoff and Beyth, 1975; Pohl, 2007), people answer questions with numbers, later see the true values for some of them, and recall their first answers. Recalls of the questions whose true values were shown land closer to those values than the first answers did, and recalls of the others do not shift that way (Pohl, Bayen, Arnold, Auer and Martin, 2018).
More about the paradigm
Here the questions are pictures, so the true values are known by construction and need no general knowledge, units or language. In part 1 each picture shows many copies of one object for 2 seconds, too briefly to count, and the participant types an estimate. A session has 20 pictures (12 to 24 as a setting), each of a different object, with about 20 to 60 copies, after 2 practice pictures that show the true number. In part 2 each object comes back in a new order and the participant types the estimate they gave. For half the pictures, matched on the number of copies, the screen also says how many there really were, printed with the request to recall as in Pohl and colleagues (2018), and the instructions ask people not to let that number change their answer, as theirs did. Part 1 does not say that part 2 will ask for the estimates.
The main score is the hindsight bias index. For each picture, the shift is how much closer to the true number the recall is than the estimate was, as a share of the estimate's distance from it: 1 when the recall is the true number, 0 when it is as far as the estimate, and down to -1. The index is the mean shift on pictures whose true number was shown minus the mean on the others, so 0 means the true numbers did not move the recalls. The shares of exact recalls and the index over the other recalls separate remembering from reconstructing, as Pohl and colleagues (2018) analysed them, and the share of shown pictures recalled as their true number counts recalls that took the number over. Pictures estimated exactly right have no shift and are left out, as are slips and answers under 100 ms. Scores support research and do not diagnose.
Scores
- Hindsight Bias Indexscore
- Mean shift toward the true number on pictures whose true number part 2 showed, minus the mean shift on the others. A shift is how much closer the recall is to the true number than the estimate was, as a share of the estimate's distance, from 1 down to -1.
- 0: the true numbers did not move the recalls; above 0: recalls moved toward the true numbers shown
- Shift With the True Numberscore
- Mean shift on pictures whose true number part 2 showed.
- Informational
- Shift Without the True Numberscore
- Mean shift on pictures whose true number part 2 did not show.
- Informational
- Exact Recalls With the True Number%
- Share of pictures with the true number shown whose recall equals the estimate.
- Informational
- Exact Recalls Without the True Number%
- Share of pictures without the true number whose recall equals the estimate.
- Informational
- Index Over Changed Recallsscore
- The hindsight bias index over recalls that differ from the estimate only.
- Informational
- Recalled as the True Number%
- Share of pictures with the true number shown whose recall equals that number.
- Informational
- Estimate Error%
- Median distance of the part 1 estimates from the true numbers, as a percent of them.
- Lower is closer
- Median Recall Timems
- Median time from a part 2 screen appearing to the recall.
- Informational
- Anticipationscount
- Pictures with an estimate or a recall submitted under 100 ms after its screen appeared, left out of the scores.
- 0 expected
- Slipscount
- Pictures whose estimate or recall was under a quarter or over four times the true number; they are left out.
- More than a quarter of the pictures is flagged
- Exact Estimatescount
- Pictures estimated exactly right in part 1; they have no shift and are left out of the index.
- Informational
- Pictures Recalledtrials
- Pictures with both an estimate and a recall, out of 20 as standard.
- Informational
Use and evidence
280 papers from 2021 to 2025 name this task in their title or abstract, down 2% compared with 2016 to 2020. That ranks it 99 of 495 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 65 |
| 2024 | 43 |
| 2023 | 61 |
| 2022 | 55 |
| 2021 | 56 |
| 2020 | 62 |
| 2019 | 55 |
| 2018 | 57 |
| 2017 | 57 |
| 2016 | 55 |
| 2015 | 48 |
| 2014 | 45 |
| 2013 | 46 |
| 2012 | 58 |
| 2011 | 47 |
| 2010 | 50 |
| 2009 | 46 |
| 2008 | 43 |
| 2007 | 52 |
| 2006 | 34 |
| 2005 | 23 |
| 2004 | 24 |
| 2003 | 49 |
| 2002 | 31 |
| 2001 | 22 |
| 2000 | 25 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 68
- Registered clinical studies using it as an outcome
- 4
Note on the counts: includes legal and clinical papers that run no task; most of its 4 trial listings are off-topic.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.22.0, counted 3 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Pictures
- 20 (standard; 12 to 24), each of a different object drawn from 24 Noto Emoji pictures, with about 20 to about 60 copies placed by our code. Each shows for 2 seconds, too briefly to count, in a circle 300 pixels across.
- Part 1
- The participant types an estimate of how many copies each picture showed, a whole number from 1 to 999, with the object shown above the question. There is no time limit, and no feedback after the practice. Part 1 does not say that part 2 will ask for the estimates.
- Part 2
- Each object comes back in a new order, never opening with the last picture of part 1, and the participant types the estimate they gave. For half the pictures, paired on the number of copies, the screen also says how many there really were, and the instructions ask people not to let that number change their answer.
- Practice
- 2 pictures (16 and 64 copies) before part 1, each followed by the true number.
- Scores
- The hindsight bias index: the mean shift toward the true number on pictures whose number was shown, minus the mean on the others. A shift is how much closer the recall is to the true number than the estimate was, as a share of the estimate's distance, from 1 down to -1. Also the exact recalls with and without the true number, the index over changed recalls, the share recalled as the true number and the error of the estimates.
- Checks
- Pictures estimated exactly right are left out of the shifts. An estimate or recall under a quarter or over 4 times the true number is a slip and is left out. Fewer than half the pictures recalled is invalid. Slips on more than a quarter of the pictures and no index are flagged and never exclude.
Studies of the memory design ask almanac questions and wait 25 minutes to a week before the recall (Pohl and colleagues, 2018). Here the questions are pictures, so no answer rests on general knowledge, units or culture, and part 2 follows part 1 within minutes, so more estimates may be recalled exactly. These are research measures of memory for estimates: no result calls anyone biased, and they do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: the scored run's part 2 rows, each a recall of a part 1 estimate. No recall is right or wrong against chance, so accuracy is not judged against chance, and recalling every estimate exactly or giving every true number back does not invalidate. Fewer than half the planned pictures estimated and recalled is invalid, and any other incomplete run is flagged. An estimate or recall under a quarter or over 4 times the true number is a slip and is left out; slips on more than a quarter of the pictures and no index are flagged for review and never exclude. Estimates and recalls under 100 ms are counted as anticipations and left out, and recalls under 200 ms on at least half the pictures invalidate.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. Hindsight bias is a shift in what people remember, so neither remembering exactly nor taking the true number is wrong in itself, and both stay in the scores. An estimate far outside the picture's range is most often a typing slip, as in the advice taking task, so it is left out; many slips can still be a real way of answering, so they flag and never exclude. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Pictures | 20 | 12 to 24 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Hindsight Bias Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Fischhoff B, 1975; Fischhoff B, Beyth R, 1975), with the standard settings (pictures: 20). The main score was hindsight bias index (mean shift toward the true number on pictures whose true number part 2 showed, minus the mean shift on the others).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Fischhoff, B. (1975). Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty. Journal of Experimental Psychology: Human Perception and Performance, 1(3), 288-299. https://doi.org/10.1037/0096-1523.1.3.288
- Fischhoff, B., & Beyth, R. (1975). I knew it would happen: Remembered probabilities of once-future things. Organizational Behavior and Human Performance, 13(1), 1-16. https://doi.org/10.1016/0030-5073(75)90002-1The memory design: people recall their own earlier judgments.
- Pohl, R. F. (2007). Ways to assess hindsight bias. Social Cognition, 25(1), 14-31. https://doi.org/10.1521/soco.2007.25.1.14The memory design and its measures.
- Pohl, R. F., Bayen, U. J., Arnold, N., Auer, T.-S., & Martin, C. (2018). Age differences in processes underlying hindsight bias: A life-span study. Journal of Cognition and Development, 19(3), 278-300. https://doi.org/10.1080/15248372.2018.1476356The true value printed with the request to recall, the instruction to remember one's own answer, and exact recalls analysed apart.
- Blank, H., Musch, J., & Pohl, R. F. (2007). Hindsight bias: On being wise after the event. Social Cognition, 25(1), 1-9. https://doi.org/10.1521/soco.2007.25.1.1The forms of hindsight bias and how they are studied.
Source papers
- Pohl, Bayen, Arnold, Auer and Martin (2018): with 48 almanac questions and a 25-minute interval, the mean shift (Δz) of recalls that were not exact was .012 without the correct answer and .188 and .276 with it read aloud or also printed; about half of all recalls were exact.Paradigm source, Children and adults
Questions
How long does the Hindsight Bias Task take?
About 6 to 8 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Hindsight Bias Index, Shift With the True Number, Shift Without the True Number, Exact Recalls With the True Number, Exact Recalls Without the True Number, Index Over Changed Recalls, Recalled as the True Number, Estimate Error, Median Recall Time, Anticipations, Slips, Exact Estimates and Pictures Recalled. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Hindsight Bias Task used in research?
280 papers from 2021 to 2025 name it in their title or abstract, down 2% compared with 2016 to 2020. That ranks it 99 of 495 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.22.0, counted 3 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Hindsight Bias Task have?
In the published research, 0 papers report its test-retest reliability and 4 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Pictures (12 to 24). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.