Reasoning
Explanatory Depth Task
Also called: illusion of explanatory depth task, IOED task, IOED paradigm, Rozenblit and Keil paradigm, understanding self-rating task
Measure the illusion of explanatory depth: participants rate how well they understand everyday devices, explain some of them in writing, step by step, and rate again.
8-14 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 26
- Evidence grade
- D
- Registered clinical studies
- 0
What it measures
Measures the illusion of explanatory depth. Participants rate how well they understand how 12 everyday devices work, from 1 to 7, then explain in writing how some of them work, step by step, and rate again after each explanation and after a question about each device. The main score is how far the ratings fall after explaining. The typed explanations are kept for the study and never scored.
Constructs
- Illusion of explanatory depth
- Metacognition
- Calibration of understanding
- Causal explanation
Research uses
- Metacognition research
- Judgment and decision-making research
- Science communication and education research
- Within-study group comparison
How the paradigm works
The explanatory depth task measures the illusion of explanatory depth: people feel they understand how everyday things work in more detail than they can explain. Rozenblit and Keil (2002) had people learn a scale from 1 to 7 with worked examples, rate how well they understood how devices such as a zipper or a flush toilet work, write a step-by-step causal explanation of some of them, answer a diagnostic question about each, and rate again. In their Study 2, as their Study 5 reports it, 33 students' mean ratings fell from 3.89 before explaining to 3.10 after explaining and 2.49 after the question, and independent raters judged the explanations at 2.72; the fall was small or absent for facts, procedures and the plots of films. Online, Meyers and colleagues (2023) found that ratings also fell for devices that were not explained. Here, each of 12 everyday devices chosen for this task, such as a refrigerator or an escalator, none of them among Rozenblit and Keil's, is rated from 1 (very little) to 7 (thoroughly), after a screen that explains the scale with a flashlight. Two of the devices (a setting, 1 to 4), drawn from the session's seed, are then explained in writing, step by step, each followed by a second rating; the other devices are rated a second time; then one diagnostic question about each explained device, such as why a touch screen does not respond to a finger in an ordinary glove, is answered in writing, each followed by a third rating. The devices and the questions are written in each of the eight languages. There is no time limit.
More about the paradigm
The main score is the drop in ratings after explaining: the first rating minus the rating right after the explanation, averaged over the explained devices, so a positive score is the illusion. The drop after the question, the drop for the devices not explained and the mean ratings at each point come with it. The typed explanations and answers are saved on their rows for the study and never scored; only very short texts on most screens are flagged. Rozenblit and Keil's expert explanations and the ratings after them are left out, as their Study 4 left them out. Scores describe ratings of one's own understanding; they are research measures and do not diagnose.
Scores
- Drop After Explainingscore
- Over the explained devices, the first rating minus the rating right after the typed explanation, on the scale from 1 to 7.
- Positive means understanding was rated lower once an explanation was tried, the illusion of explanatory depth
- Drop After the Questionscore
- Over the explained devices, the first rating minus the rating after the diagnostic question.
- Positive expected: the question tests the knowledge the explanation claimed
- Drop for Devices Not Explainedscore
- Over the devices not explained, the first rating minus the second, made after the explanations.
- How far ratings fall without explaining that device
- Mean First Ratingscore
- Mean of the first ratings of all the devices, from 1 to 7.
- Informational
- First Rating of Explained Devicesscore
- Mean of the first ratings of the devices later explained.
- Informational
- Rating After Explainingscore
- Mean rating of the explained devices right after their explanation.
- Informational
- Rating After the Questionscore
- Mean rating of the explained devices after their diagnostic question.
- Informational
- Median Rating Timems
- Median time from a rating screen appearing to a rating, over ratings at or after 100 ms.
- Informational
- Ratings Under 100 mscount
- Ratings made less than 100 ms after their screen appeared.
- 0 expected
- Devices Explainedcount
- Explained devices rated again after their explanation.
- Informational
Use and evidence
26 papers from 2021 to 2025 name this task in their title or abstract, up 8% compared with 2016 to 2020. That ranks it 378 of 495 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 5 |
| 2024 | 8 |
| 2023 | 5 |
| 2022 | 4 |
| 2021 | 4 |
| 2020 | 7 |
| 2019 | 6 |
| 2018 | 4 |
| 2017 | 2 |
| 2016 | 5 |
| 2015 | 4 |
| 2014 | 3 |
| 2013 | 1 |
| 2012 | 1 |
| 2011 | 1 |
| 2010 | 2 |
| 2009 | 1 |
| 2008 | 2 |
| 2007 | 0 |
| 2006 | 0 |
| 2005 | 0 |
| 2004 | 1 |
| 2003 | 2 |
| 2002 | 1 |
| 2001 | 0 |
| 2000 | 0 |
Use grade E
Fewer than 30 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 10
- Registered clinical studies using it as an outcome
- 0
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.22.0, counted 3 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Devices
- 12 everyday devices chosen for this task, such as a refrigerator, an escalator and a seat belt, none of them among Rozenblit and Keil's, each with one diagnostic question. They are shown in the session's language.
- Rating
- How well do you understand how the device works, from 1 (very little) to 7 (thoroughly), on seven buttons, after a screen that explains the scale with a flashlight.
- Procedure
- All 12 rated in an order drawn from the session's seed. Then 2 devices by default, 1 to 4 as a setting, drawn from the seed: each explained in writing, step by step, then rated again. Then the other devices rated a second time. Then one diagnostic question per explained device, answered in writing, each followed by a third rating.
- Typed text
- Explanations and answers of up to 2,000 characters, with pasting blocked. They are saved for the study and never scored.
- Scores
- The drop in ratings after explaining: the first rating minus the rating right after the explanation, over the explained devices. Also the drop after the question, the drop for the devices not explained and the mean ratings.
Rozenblit and Keil (2002) found that ratings fell after people tried to explain devices, and that independent raters judged the explanations about as low as the ratings people gave after the diagnostic question. Meyers and colleagues (2023) found online that ratings also fell for devices that were not explained, and fell half as much when nothing was explained, so this task also rates the other devices again. Rozenblit and Keil's expert explanations and the ratings after them are left out, as their Study 4 left them out. The scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: the scored run's ratings of the devices. No rating is correct, so accuracy is not judged against chance. A run with no device rated again after an explanation, or with first ratings for fewer than half the devices, is invalid, and any other shortfall is flagged; the same first rating for every device and typed texts under 10 characters on at least half of them are flagged. Ratings under 200 ms on at least half the screens invalidate.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. The scores describe how a person's ratings of their own understanding change after trying to explain, so any rating is a possible answer, and so is a rating that does not fall. The drop needs an explanation to have been tried; explanations are never scored, and only very short texts on most screens are flagged, because "I do not know" is a real answer. One rating for every device suggests the scale was not used; it is flagged for review because it can also be a real answer. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Devices explained | 2 | 1 to 4 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Explanatory Depth Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Rozenblit L, Keil F, 2002), with the standard settings (devices explained: 2). The main score was drop after explaining (over the explained devices, the first rating minus the rating right after the typed explanation, on the scale from 1 to 7).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Rozenblit, L., & Keil, F. (2002). The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive Science, 26(5), 521-562. https://doi.org/10.1207/s15516709cog2605_1
- Meyers, E. A., Gretton, J. D., Budge, J. R. C., Fugelsang, J. A., & Koehler, D. J. (2023). Broad effects of shallow understanding: Explaining an unrelated phenomenon exposes the illusion of explanatory depth. Judgment and Decision Making, 18, e24. https://doi.org/10.1017/jdm.2023.24Ratings of devices not explained, online.
- Fernbach, P. M., Rogers, T., Fox, C. R., & Sloman, S. A. (2013). Political extremism is supported by an illusion of understanding. Psychological Science, 24(6), 939-946. https://doi.org/10.1177/0956797612464058Mechanistic explanations lower self-rated understanding.
- Alter, A. L., Oppenheimer, D. M., & Zemla, J. C. (2010). Missing the trees for the forest: A construal level account of the illusion of explanatory depth. Journal of Personality and Social Psychology, 99(3), 436-451. https://doi.org/10.1037/a0020218Why the illusion arises.
Source papers
- Rozenblit and Keil (2002), Study 2 as their Study 5 reports it: 33 undergraduates' mean ratings were 3.89 before explaining, 3.10 after explaining and 2.49 after the diagnostic question, on a scale from 1 to 7, and independent raters judged their explanations at 2.72.Paradigm source, Adults
Questions
How long does the Explanatory Depth Task take?
About 8 to 14 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Drop After Explaining, Drop After the Question, Drop for Devices Not Explained, Mean First Rating, First Rating of Explained Devices, Rating After Explaining, Rating After the Question, Median Rating Time, Ratings Under 100 ms and Devices Explained. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Explanatory Depth Task used in research?
26 papers from 2021 to 2025 name it in their title or abstract, up 8% compared with 2016 to 2020. That ranks it 378 of 495 test families in the ConductCognition test census (use grade E). Source: ConductCognition test census 0.22.0, counted 3 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Explanatory Depth Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Devices explained (1 to 4). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.