Reasoning
Changing Evidence Task
Also called: bias against disconfirmatory evidence task, BADE task, evidence integration task, belief flexibility task, disconfirmatory evidence task
Measure how people revise beliefs as evidence arrives: short stories unfold one sentence at a time, and four explanations are rated after each sentence.
10-15 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 33
- Evidence grade
- C
- Registered clinical studies
- 6
What it measures
Measures how people revise their ratings of explanations as evidence arrives. Participants read 12 short everyday stories one sentence at a time and, after each of three sentences, rate how likely four explanations are, from 0 to 10. Two explanations fit the first sentence and are contradicted later; the main score is how far their ratings fall from the first sentence to the last. A research measure with no clinical claim.
Constructs
- Belief revision
- Evidence integration
- Plausibility judgment
Research uses
- Psychosis and delusion research
- Reasoning and belief-updating research
- Within-study group comparison
How the paradigm works
The changing evidence task measures how people revise their ratings of explanations as evidence arrives. It follows the bias against disconfirmatory evidence (BADE) task of Woodward, Moritz, Cuttler and Whitman (2006), in the written form of Woodward, Moritz, Menon and Klinge (2008): a scenario unfolds step by step, and after each step four explanations are rated for plausibility. One explanation is the one the story turns out to support; two, one neutral and one that names a feeling, fit the start but are contradicted later; and one is far-fetched throughout.
More about the paradigm
The participant reads 12 short everyday stories (6 to 12 as a setting), written for this task in each of the eight languages, one sentence at a time over three sentences. After each sentence they rate each of the four explanations from 0 (very unlikely) to 10 (very likely), each on its own; at the second and third sentences their earlier ratings are already chosen and can be changed. The explanations keep their places within a story, in an order drawn from the session's seed, and the stories come in a seeded order. One practice story comes first. There is no time limit and no feedback.
Scores
- Drop in Contradicted Explanationspoints
- Mean rating of the two explanations the later sentences contradict, after the first sentence less after the third, averaged over stories: the BADE score of the literature.
- Higher means ratings followed the contradicting evidence further
- Rise in the Supported Explanationpoints
- Rating of the explanation the story supports, after the third sentence less after the first, averaged over stories.
- Higher means ratings followed the supporting evidence further
- Far-Fetched Explanation Ratingpoints
- Mean rating, from 0 to 10, of the far-fetched explanation over all three sentences: liberal acceptance in the literature.
- Higher means far-fetched explanations were rated more likely
- Drop in the Neutral Contradicted Explanationpoints
- The drop for the contradicted explanation that names no feeling.
- Informational
- Drop in the Feeling Contradicted Explanationpoints
- The drop for the contradicted explanation that names a feeling.
- Informational
- Contradicted Explanations by Sentencepoints
- Mean rating of the contradicted explanations after each sentence, keyed by step.
- Falls across the steps as the evidence arrives
- Supported Explanation by Sentencepoints
- Mean rating of the supported explanation after each sentence, keyed by step.
- Rises across the steps as the evidence arrives
- Ratings Revised%
- Share of ratings at the second and third sentences that differ from the step before.
- 0 means no rating ever changed, which is flagged for review
- Scale Points Usedcount
- How many of the eleven points from 0 to 10 the ratings used.
- 1 means one rating everywhere, which is flagged for review
- Practice Check0/1
- 1 when the practice story's final ratings put the supported explanation above the far-fetched one.
- 0 is flagged for review
- Median Step Timems
- Median time from a step's screen to Continue.
- Informational; there is no time limit
- Fast Stepscount
- Steps confirmed under 1,000 ms after their screen opened.
- Half or more of the steps makes the session invalid
- Stories Completedcount
- Stories with all four ratings after all three sentences in the scored run.
- Under 90 percent of those planned makes the session invalid
- Stories Plannedcount
- Stories the run set out to show.
- Informational
Use and evidence
33 papers from 2021 to 2025 name this task in their title or abstract, up 18% compared with 2016 to 2020. That ranks it 420 of 582 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 12 |
| 2024 | 5 |
| 2023 | 6 |
| 2022 | 3 |
| 2021 | 7 |
| 2020 | 6 |
| 2019 | 2 |
| 2018 | 9 |
| 2017 | 5 |
| 2016 | 6 |
| 2015 | 3 |
| 2014 | 6 |
| 2013 | 2 |
| 2012 | 7 |
| 2011 | 2 |
| 2010 | 2 |
| 2009 | 1 |
| 2008 | 2 |
| 2007 | 3 |
| 2006 | 4 |
| 2005 | 0 |
| 2004 | 0 |
| 2003 | 1 |
| 2002 | 0 |
| 2001 | 0 |
| 2000 | 1 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 9
- Registered clinical studies using it as an outcome
- 6
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.39.0, counted 10 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Stories
- 12 written for this test (standard, 6 to 12), in the session's language: everyday scenes with no politics, health, religion or groups, and people named by role. A practice story comes first.
- Steps
- Each story arrives in 3 sentences, one per step. After each sentence the participant rates each of 4 explanations from 0 (very unlikely) to 10 (very likely), each on its own. At steps 2 and 3 the earlier ratings are already chosen and can be changed.
- Explanations
- One the story turns out to support; two that fit the first sentence and are contradicted by the later ones, one neutral and one naming a feeling; and one far-fetched throughout. They keep their places within a story, in an order drawn from the session's seed.
- Order
- The stories come in an order drawn from the session's seed. There is no time limit and no feedback.
- Score
- The mean rating of the two contradicted explanations after the first sentence less after the third, averaged over stories: the BADE score. The supported explanation's rise, the far-fetched explanation's mean rating, each contradicted explanation's drop and the mean ratings after each sentence come with it.
Known in the literature as the bias against disconfirmatory evidence (BADE) task. Its scores describe ratings of explanations in stories written for this task and support research; the task makes no clinical claim. Every session uses the same stories, so a second session can carry what the first one showed. No percentile is given.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: changing_evidence_step from the scored run, four ratings from 0 to 10 after each of a story's three sentences, untimed, with no correct rating. Fewer than 90% of the planned stories with every rating invalidate. Steps confirmed under 1,000 ms invalidate when they are at least half of four or more and are flagged otherwise. The same rating everywhere, no rating changed after any first sentence, or a practice story ending with the far-fetched explanation at or above the supported one is flagged for review. A hidden tab is flagged.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. Ratings that never change can be sincere, so they are flagged, never treated as wrong; holding to the first ratings is what the task measures (Woodward, Moritz, Cuttler and Whitman 2006). The practice checks use of the scale, not a belief. A step needs the new sentence read before Continue, so a step confirmed within 1,000 ms cannot reflect it; following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the steps are that fast. The 90% completion threshold is a ConductScience choice.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Stories | 12 | 6 to 12 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Changing Evidence Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Woodward, Moritz, Cuttler and Whitman, 2006; Woodward, Moritz, Menon and Klinge, 2008), with the standard settings (stories: 12). The main score was drop in contradicted explanations (mean rating of the two explanations the later sentences contradict, after the first sentence less after the third, averaged over stories: the BADE score of the literature).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Woodward, T. S., Moritz, S., Cuttler, C., & Whitman, J. C. (2006). The contribution of a cognitive bias against disconfirmatory evidence (BADE) to delusions in schizophrenia. Journal of Clinical and Experimental Neuropsychology, 28(4), 605-617. https://doi.org/10.1080/13803390590949511The paradigm: four interpretations of a scenario rated for plausibility as new information arrives, two tenable at first and then implausible, one untenable at first and then plausible, and one untenable throughout.
- Woodward, T. S., Moritz, S., Menon, M., & Klinge, R. (2008). Belief inflexibility in schizophrenia. Cognitive Neuropsychiatry, 13(3), 267-277. https://doi.org/10.1080/13546800802099033The written form: statements that give increasingly detailed information about a scenario, with the true interpretation gaining plausibility and the lures losing it.
- Tenconi, E., Meregalli, V., Buffa, A., Collantoni, E., Cavallaro, R., Meneguzzo, P., & Favaro, A. (2023). Belief inflexibility and cognitive biases in anorexia nervosa: The role of the bias against disconfirmatory evidence and its clinical and neuropsychological correlates. Journal of Clinical Medicine, 12(5), 1746. https://doi.org/10.3390/jcm12051746Three statements per scenario, a neutral and an emotional lure, the 0 to 10 scale, and the BADE, BACE and liberal acceptance scores.
Source papers
- Woodward, Moritz, Cuttler and Whitman (2006) and Woodward, Moritz, Menon and Klinge (2008): scenarios disambiguated step by step, with true, lure and absurd explanations rated after each step. Tenconi et al. (2023): the BADE, BACE and liberal acceptance scores on a 0 to 10 scale.Paradigm source, Adults
Questions
How long does the Changing Evidence Task take?
About 10 to 15 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Drop in Contradicted Explanations, Rise in the Supported Explanation, Far-Fetched Explanation Rating, Drop in the Neutral Contradicted Explanation, Drop in the Feeling Contradicted Explanation, Contradicted Explanations by Sentence, Supported Explanation by Sentence, Ratings Revised, Scale Points Used, Practice Check, Median Step Time, Fast Steps, Stories Completed and Stories Planned. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Changing Evidence Task used in research?
33 papers from 2021 to 2025 name it in their title or abstract, up 18% compared with 2016 to 2020. That ranks it 420 of 582 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.39.0, counted 10 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Changing Evidence Task have?
In the published research, 0 papers report its test-retest reliability and 6 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Stories (6 to 12). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.