Reasoning
Reasoning Biases Problems
Also called: heuristics and biases tasks, conjunction fallacy problems, base-rate neglect problems, ratio bias task, denominator neglect, covariation detection
Measure how often people give the normative answer when intuition points elsewhere, on conjunction, base-rate, ratio-bias and covariation problems written for this test.
7-10 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 198
- Evidence grade
- C
- Registered clinical studies
- 0
What it measures
Measures how often people give the normative answer when an intuitive cue points elsewhere. Participants choose one of two answers on 24 short problems of four classic kinds: conjunction, base rate, ratio bias and covariation. The main score counts normative answers on the 16 problems where intuition and the rule disagree.
Constructs
- Judgment under uncertainty
- Bias susceptibility
- Probabilistic reasoning
- Analytic thinking
Research uses
- Judgment and decision-making research
- Dual-process and conflict detection research
- Individual differences in rational thinking
- Within-study group comparison
Scores
- Normative Answers, Conflict Problemscount
- Conflict problems answered with the normative answer, out of 16.
- Higher means intuition was overridden more often
- Conjunction Problemscount
- Conjunction conflict problems answered with the single statement, out of 4.
- Higher means fewer conjunction errors
- Base-Rate Problemscount
- Base-rate conflict problems answered with the larger group, out of 4.
- Higher means base rates were used more
- Ratio Problemscount
- Ratio conflict problems answered with the container that has the higher share of winners, out of 4.
- Higher means shares were compared rather than counts
- Covariation Problemscount
- Covariation conflict problems answered by comparing the shares of good outcomes, out of 4.
- Higher means both groups were compared
- Normative Answers, Control Problemscount
- Control problems, where intuition and the rule agree, answered normatively, out of 8.
- Expected near 8; fewer than 75% is flagged for review
- Conflict Slowingms
- Median time of intuitive answers on conflict problems minus the median time of normative answers on control problems. Needs 2 answers in each.
- Positive means slower on conflict problems even when answering intuitively, a sign of noticing the conflict
- Median Decision Timems
- Median time from a problem's screen to the answer, over answers at or after 100 ms.
- Describes pace; the test has no time limit
- Answers Under 100 mscount
- Answers given less than 100 ms after the screen appeared. They still count as answers.
- Expected to be 0
- Problems Answeredcount
- Problems answered in the scored run.
- 24 in a full session
Use and evidence
198 papers from 2021 to 2025 name this task in their title or abstract, up 9% compared with 2016 to 2020. That ranks it 123 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 50 |
| 2024 | 38 |
| 2023 | 34 |
| 2022 | 38 |
| 2021 | 38 |
| 2020 | 39 |
| 2019 | 35 |
| 2018 | 37 |
| 2017 | 37 |
| 2016 | 34 |
| 2015 | 34 |
| 2014 | 24 |
| 2013 | 24 |
| 2012 | 25 |
| 2011 | 24 |
| 2010 | 13 |
| 2009 | 29 |
| 2008 | 22 |
| 2007 | 35 |
| 2006 | 9 |
| 2005 | 13 |
| 2004 | 6 |
| 2003 | 13 |
| 2002 | 10 |
| 2001 | 4 |
| 2000 | 12 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 1
- Papers on reliability or validity
- 39
- Registered clinical studies using it as an outcome
- 0
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Problems
- 24 written for this test, 6 of each kind: conjunction, base rate, ratio bias and covariation. They are shown in the session's language, with the same numbers in every language.
- Conflict and control
- Each kind has 4 conflict problems, where the intuitive answer and the normative answer differ, and 2 control problems, where they agree.
- Conjunction
- A short description of a person, then a single statement and the same statement joined with a second one. The single statement is the normative answer.
- Base rate
- A group in which at least 99 in 100 people belong to one part, and one person picked at random. The larger part is the normative answer, whichever part the description fits.
- Ratio bias
- A small and a large container, for example 1 winner in 10 against 9 in 100. The higher share of winners is the normative answer, not the larger number of winners.
- Covariation
- A new method tried on some cases and not on others, with the good outcomes of each. It helped only if the share of good outcomes is higher with it.
- Choice
- Click or tap one of two answers, with no time limit and no feedback. The order, with no more than 2 problems of one kind in a row, and which answer comes first are drawn from the session's seed; the normative answer comes first on half the problems of each kind.
- Practice
- 1 problem with no conflict, followed by its answer.
- Scores
- Normative answers on the 16 conflict problems, overall and for each kind; normative answers on the 8 control problems; and how much slower intuitive answers to conflict problems come than answers to control problems.
The problems follow the classic structures but were written for this test; no published problem, such as the Linda problem, is used. Control problems are answered correctly by nearly everyone who reads them, so they check attention. The scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: the scored run's answers to the 24 problems. Only the 8 control problems, where intuition and the rule agree, are judged against chance (50%): below chance invalidates, and fewer than 75% normative answers on them is flagged. One answer position on 90% of problems invalidates when the control answers are not above chance. A run with fewer than half its problems answered is invalid and any other shortfall is flagged. Answers under 200 ms on at least half the problems invalidate.
Why these rules
On conflict problems the intuitive answer is the bias the test measures, so a low or below-chance score there is a result, not a sign of careless answering. Control problems need no resistance to intuition, so failing them means the problems were not read (De Neys and Glumicic 2008 found near-ceiling accuracy on such problems). The normative answer is first on half the problems of each kind, so pressing one position cannot follow the problems. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
Running this test
Settings you can change
This test runs with fixed settings.
Need a different version of this test?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Reasoning Biases Problems task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Tversky & Kahneman, 1983; Kahneman & Tversky, 1973). The main score was normative Answers, conflict problems (conflict problems answered with the normative answer, out of 16).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4), 293-315. https://doi.org/10.1037/0033-295X.90.4.293
- Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237-251. https://doi.org/10.1037/h0034747
- Denes-Raj, V., & Epstein, S. (1994). Conflict between intuitive and rational processing: When people behave against their better judgment. Journal of Personality and Social Psychology, 66(5), 819-829. https://doi.org/10.1037/0022-3514.66.5.819
- Wasserman, E. A., Dorner, W. W., & Kao, S.-F. (1990). Contributions of specific cell information to judgments of interevent contingency. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(3), 509-521. https://doi.org/10.1037/0278-7393.16.3.509
- De Neys, W., & Glumicic, T. (2008). Conflict monitoring in dual process theories of thinking. Cognition, 106(3), 1248-1299. https://doi.org/10.1016/j.cognition.2007.06.002Conflict and control problems, and slower answers on conflict problems.
- De Neys, W., Cromheeke, S., & Osman, M. (2011). Biased but in doubt: Conflict and decision confidence. PLoS ONE, 6(1), e15954. https://doi.org/10.1371/journal.pone.0015954Two-answer conjunction and base-rate problems with conflict and no-conflict versions.
- Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39(7), 1275-1289. https://doi.org/10.3758/s13421-011-0104-1Heuristics and biases problems as measures of individual differences.
- Berthet, V., & de Gardelle, V. (2023). The heuristics-and-biases inventory: An open-source tool to explore individual differences in rationality. Frontiers in Psychology, 14, 1145246. https://doi.org/10.3389/fpsyg.2023.1145246Reliability of bias measures with several problems each.
Source papers
- De Neys W, Cromheeke S, Osman M (2011). Biased but in doubt: conflict and decision confidence. PLoS ONE, 6(1):e15954.Paradigm source, Adults
Questions
How long does the Reasoning Biases Problems take?
About 7 to 10 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Normative Answers, Conflict Problems, Conjunction Problems, Base-Rate Problems, Ratio Problems, Covariation Problems, Normative Answers, Control Problems, Conflict Slowing, Median Decision Time, Answers Under 100 ms and Problems Answered. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Reasoning Biases Problems used in research?
198 papers from 2021 to 2025 name it in their title or abstract, up 9% compared with 2016 to 2020. That ranks it 123 of 395 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Reasoning Biases Problems have?
In the published research, 1 paper reports its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.