Reasoning
Syllogistic Reasoning Task
Also called: belief bias task, syllogism task, conclusion evaluation task
Measure logical reasoning and belief bias: participants judge whether conclusions must follow, with validity and believability crossed.
8-10 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 232
- Evidence grade
- C
- Registered clinical studies
- 0
What it measures
Measures logical reasoning and how far belief pulls on it. Participants read two statements and a conclusion and answer whether the conclusion must be true, on 32 problems that cross valid and invalid logic with believable and unbelievable conclusions. The problems are in English.
Constructs
- Deductive reasoning
- Belief bias
- Analytic thinking
Research uses
- Reasoning and judgment research
- Individual differences in belief bias
- Within-study group comparison
How the paradigm works
The syllogistic reasoning task measures logical reasoning and how far prior belief pulls on it (Evans, Barston and Pollard 1983). Each problem shows two statements and a conclusion. Participants assume the statements are true and answer whether the conclusion must be true. Validity and believability are crossed: 32 problems, 8 each of valid believable, valid unbelievable, invalid believable and invalid unbelievable.
More about the paradigm
Eight forms in 4 matched pairs are used, all concluding "Some C are not A". The two forms of a pair share their quantifiers and figure and differ only in which end term sits in which premise. The contents were written for this task: a category, a kind within it and a neutral place. By general knowledge, "Some fruits are not apples" is believable and "Some apples are not fruits" is unbelievable. The session's random seed deals the contents to forms and believability, so a conclusion is not tied to one condition, and orders the problems; both are recorded. Two practice problems outside the set have feedback. The test has no feedback and no time limit. The problems are English; sessions run in other languages are flagged for review.
The logic index is the mean Yes rate on valid problems minus the mean on invalid problems. The belief index is the mean Yes rate on believable conclusions minus the mean on unbelievable ones. The interaction index is (VB - IB) - (VU - IU), the sign Trippas and colleagues (2018) use: negative values mean the gap between valid and invalid problems is smaller for believable conclusions, the usual pattern, and some papers report the opposite sign. Klauer, Musch and Naumer (2000) and Dube, Rotello and Heit (2010) show that this index can arise from response bias alone. With the confidence setting on, each answer is followed by a sureness rating, which gives the six-point scale of Dube and colleagues and the area under the empirical ROC for believable and unbelievable conclusions. Scores support research and do not diagnose.
Scores
- Percent Correct%
- Percent of problems answered correctly: Yes on valid problems and No on invalid ones.
- Higher means more problems solved by their logic; 50% is chance
- Logic Indexscore
- Mean Yes rate on valid problems minus the mean on invalid problems.
- Higher means answers followed validity more
- Belief Indexscore
- Mean Yes rate on believable conclusions minus the mean on unbelievable ones.
- Higher means answers followed believability more
- Interaction Indexscore
- The Yes rate gap between valid and invalid problems for believable conclusions, minus the same gap for unbelievable ones.
- Negative means validity moved answers less for believable conclusions; can reflect response bias
- Yes Rate, Valid Believable%
- Share of valid problems with believable conclusions answered Yes.
- Higher means more accepted
- Yes Rate, Valid Unbelievable%
- Share of valid problems with unbelievable conclusions answered Yes.
- Higher means more accepted
- Yes Rate, Invalid Believable%
- Share of invalid problems with believable conclusions answered Yes.
- Higher means more accepted
- Yes Rate, Invalid Unbelievable%
- Share of invalid problems with unbelievable conclusions answered Yes.
- Higher means more accepted
- Median Response Timems
- Median time from a problem's appearance to its answer, leaving out answers under 100 ms.
- Describes pace; the test has no time limit
- Answers Under 100 mscount
- Problems answered within 100 ms.
- Expected to be 0
- Problems Completedtrials
- Problems answered in the scored run.
- A session with fewer than 24 is invalid
- Longest Same Answer Runtrials
- The longest run of one answer.
- 12 or more is flagged for review
- ROC Area, Believablescore
- Area under the empirical ROC for believable conclusions, from answers and sureness ratings. Empty when ratings are off.
- 0.5 is chance; higher means answers and sureness tracked validity better
- ROC Area, Unbelievablescore
- Area under the empirical ROC for unbelievable conclusions, from answers and sureness ratings. Empty when ratings are off.
- 0.5 is chance; higher means answers and sureness tracked validity better
- ROC Area Differencescore
- ROC area for believable conclusions minus the area for unbelievable ones. Empty when ratings are off.
- Negative means validity was judged better on unbelievable conclusions
Use and evidence
232 papers from 2021 to 2025 name this task in their title or abstract, up 12% compared with 2016 to 2020. That ranks it 108 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 59 |
| 2024 | 64 |
| 2023 | 47 |
| 2022 | 27 |
| 2021 | 35 |
| 2020 | 47 |
| 2019 | 44 |
| 2018 | 44 |
| 2017 | 42 |
| 2016 | 30 |
| 2015 | 25 |
| 2014 | 36 |
| 2013 | 37 |
| 2012 | 28 |
| 2011 | 38 |
| 2010 | 29 |
| 2009 | 25 |
| 2008 | 17 |
| 2007 | 19 |
| 2006 | 12 |
| 2005 | 16 |
| 2004 | 16 |
| 2003 | 10 |
| 2002 | 11 |
| 2001 | 4 |
| 2000 | 11 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 1
- Papers on reliability or validity
- 127
- Registered clinical studies using it as an outcome
- 0
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Problems
- 32 problems, 8 each of valid believable, valid unbelievable, invalid believable and invalid unbelievable.
- Forms
- 8 forms in 4 matched pairs, all concluding "Some C are not A": EIO-2, EIO-3, AOO-2 and OAO-3 are valid; IEO-2, IEO-3, OAO-2 and AOO-3 are invalid. The forms of a pair share their quantifiers and figure.
- Contents
- A category, a kind within it and a neutral place. By general knowledge, "Some fruits are not apples" is believable and "Some apples are not fruits" is unbelievable.
- Assignment
- The session's seed deals the contents to forms and believability, so a conclusion is not tied to one condition, and orders the problems with no more than 4 of one correct answer in a row. All of it is saved.
- Answer
- Yes or No to whether the conclusion must be true, with no time limit and no feedback. The side of each button is drawn from the seed and saved.
- Practice
- 2 problems outside the set, each followed by feedback.
- Confidence
- Off by default. As a setting, a rating after each answer (guessing, fairly sure or very sure) gives a six-point scale and ROC areas.
The contents and sentences were written for this task; no published problem set is used. The problems are English; sessions run in other languages are flagged for review.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: syllogistic_reasoning_problem from the scored run, one Yes or No answer per problem, half of them valid, so chance is 50%. Accuracy significantly below chance invalidates. One answer on at least 90% of problems invalidates unless accuracy is above chance. Answers under 200 ms on at least half of 4 or more problems invalidate. Fewer than 24 of the 32 problems completed invalidates. A run of 12 or more of one answer, or a session run in another language, is flagged for review. A hidden tab is flagged.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. A person who follows belief alone scores near chance, since half of each believability level is valid, so a score at chance can be a real result of the kind this test studies and is only flagged. The four cells need most of their problems for the indices to mean the same thing across sessions, so fewer than three quarters of the problems invalidates. Correct answers never run longer than 4 in a row, so a much longer run of one answer is flagged. The problems are English; a session in another language shows them to a participant who may not read English fluently, so it is flagged for review. Nothing runs while the tab is hidden, so a hidden tab is flagged.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Confidence ratings | Off | On or off |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Instructions and screens: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. Words and sentences: English.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Syllogistic Reasoning Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Evans, Barston & Pollard, 1983), with the standard settings (confidence ratings: Off). The main score was percent correct (percent of problems answered correctly: Yes on valid problems and No on invalid ones).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Evans, J. St. B. T., Barston, J. L., & Pollard, P. (1983). On the conflict between logic and belief in syllogistic reasoning. Memory & Cognition, 11(3), 295-306. https://doi.org/10.3758/BF03196976
- Klauer, K. C., Musch, J., & Naumer, B. (2000). On belief bias in syllogistic reasoning. Psychological Review, 107(4), 852-884. https://doi.org/10.1037/0033-295X.107.4.852
- Dube, C., Rotello, C. M., & Heit, E. (2010). Assessing the belief bias effect with ROCs: It's a response bias effect. Psychological Review, 117(3), 831-863. https://doi.org/10.1037/a0019634
- Trippas, D., Kellen, D., Singmann, H., Pennycook, G., Koehler, D. J., Fugelsang, J. A., & Dubé, C. (2018). Characterizing belief bias in syllogistic reasoning: A hierarchical Bayesian meta-analysis of ROC data. Psychonomic Bulletin & Review, 25(6), 2141-2174. https://doi.org/10.3758/s13423-018-1460-7
- Khemlani, S., & Johnson-Laird, P. N. (2012). Theories of the syllogism: A meta-analysis. Psychological Bulletin, 138(3), 427-457. https://doi.org/10.1037/a0026841
Source papers
- Evans JStBT, Barston JL, Pollard P (1983). On the conflict between logic and belief in syllogistic reasoning. Memory & Cognition, 11(3):295-306.Paradigm source, Adults
- Trippas D, Kellen D, Singmann H, Pennycook G, Koehler DJ, Fugelsang JA, Dubé C (2018). Characterizing belief bias in syllogistic reasoning: a hierarchical Bayesian meta-analysis of ROC data. Psychonomic Bulletin & Review, 25(6):2141-2174.Paradigm source, Adults
Questions
How long does the Syllogistic Reasoning Task take?
About 8 to 10 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Percent Correct, Logic Index, Belief Index, Interaction Index, Yes Rate, Valid Believable, Yes Rate, Valid Unbelievable, Yes Rate, Invalid Believable, Yes Rate, Invalid Unbelievable, Median Response Time, Answers Under 100 ms, Problems Completed, Longest Same Answer Run, ROC Area, Believable, ROC Area, Unbelievable and ROC Area Difference. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Syllogistic Reasoning Task used in research?
232 papers from 2021 to 2025 name it in their title or abstract, up 12% compared with 2016 to 2020. That ranks it 108 of 395 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Syllogistic Reasoning Task have?
In the published research, 1 paper reports its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Which settings can a researcher change?
Confidence ratings (On or off). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
The instructions and screens are in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. The words and sentences stay in English, so sessions in other languages are flagged for review. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.