ConductCognition
  • Domains

    Tests

    • Processing speed55
    • Attention143
    • Working memory51
    • Executive function191
    • Episodic memory84
    • Response inhibition10
    • Social cognition97
    • Language50

    Most used

    Evidence

    1. Verbal Fluency TestGradeA
    2. Stroop Color-Word TestGradeA
    3. Trail Making TestGradeA
    4. N-Back TaskGradeA
    5. Go/No-Go TaskGradeA
    6. Digit Span TestGradeA
    7. Symbol Coding TestGradeA
    8. Verbal Learning TestGradeA
    REDTry it nowStroop Color-Word Test
    All 681 tests

    Browse by

    • Phone380
    • Tablet415
    • Untimed419
    • Up to 5 minutes81
    • Evidence grade A48
  • First studies

    Grade, length

    • Reaction Time StudyGradeA7 min
    • Pilot ScreeningGradeA15 min
    • Medication StudiesGradeA−26 min
    • Language-Minimized ScreeningGradeA−30 min

    Age groups

    • Older Adult Executive SpeedGradeA20 min
    • School-AgeGradeA−33 min

    Memory

    • At-Home MemoryGradeA26 min
    • Working Memory and LearningGradeA−19 min
    • Associative MemoryGradeB32 min
    • Working Memory CapacityGradeB34 min

    Language and perception

    • LanguageGradeA−33 min
    • Visual PerceptionGradeB27 min
    • Listening and Auditory ProcessingGradeB31 min

    Executive function and attention

    • Executive ControlGradeA18 min
    • Attention and ImpulsivityGradeA−26 min
    • Visual AttentionGradeB27 min
    • Creativity and Flexible ThinkingGradeB30 min
    • Reasoning and SpatialGradeB32 min

    Movement

    • Motor and AttentionGradeA−29 min
    • Motor LearningGradeB34 min

    Behavior, mood and choice

    • Food Choice and Self-ControlGradeA−29 min
    • Sleep and FatigueGradeB23 min
    • Social CognitionGradeB23 min
    • Impulsivity and Substance UseGradeB26 min
    • Emotion and MoodGradeB28 min
    • Decisions and RewardGradeB29 min
    All 26 batteries

    Find one

    Up to 15 minutes2Up to 30 minutes19Phone8
    Test order labBattery gradesHow batteries work
  • Use and evidence

    • Overview
    • Compare all 681 tests
    • Test-retest
    • Reliability and validity
    • Battery grades26
    • Download and cite

    Clinical trials

    • Overview
    • By condition25
    • By test103
    • Trial design benchmark
    • Cognitive outcomes, 2026

    Conditions

    Registered studies

    • Dementia and Alzheimer's1,673
    • Aging, frailty and exercise1,400
    • Depression and bipolar disorder1,249
    • Stroke1,237
    • Substance use and addiction904
    • Parkinson's disease894
    • Cancer762
    All 25 conditions
    16,526Registered studies name a cognitive test in an outcome measure11,848 name a test you can run hereExplore the studies
  • 1

    Build

    • Study design
    • Customize tests
    • Repeat visits
    • Study cost calculator
    2

    Recruit

    • Recruitment setup
    • Prolific
    • Sona Systems
    • CloudResearch Connect
    • Qualtrics
    3

    Run

    • Remote testing
    • Participant devices
    • Phone timing check
    • Offline edition
    4

    Results

    • Automated scoring
    • Test norms
    • Study Quality Monitor
    • Data export
    Platform overview

    Compliance

    • HIPAA-ready testing
    • US clinical research
    • Data Policy
    • Privacy Policy
    • Terms of Use
  • Research areas

    Tests used in each

    • Aging and dementia202
    • Decisions and reward136
    • Child development and education114
    • Social behavior87
    • Language and bilingualism80
    • Psychiatric groups35
    • Training and intervention31
    • Mood, anxiety and trauma28
    • Substance use and impulsivity26
    • Neurological and movement research21
    • Sleep and fatigue17
    • Brain injury and concussion14
    • ADHD11
    • Eating and obesity11
    • Medication and drug trials8

    Research settings

    • Student participant pools
    • Teaching a class
    • Clinical trials
    • US clinical research
    • Sites without internet
    Planning a study?
  • Methods

    • How evidence grades work
    • How test batteries work
    • How test order works
    Methods and questions

    Help

    • FAQ
    • Recruitment guides
    • Compare platforms

    Updates

    • Blog
    • New tests by email
    Latest post10 November 2026What you can do in ConductCognition todayA short tour of ConductCognition as it stands on 10 November. After this, we'll post when something changes how you run a study.Read the post
  • Pricing
Start Free
Log In
Log InStart Free
  1. Home
  2. /Tests
  3. /Decision Quality Task

Reasoning

Decision Quality Task

Also called: outcome bias, outcome effect, outcome bias in decision evaluation, judging decisions by their outcomes, evaluating decisions after the fact

Measure the outcome bias: participants read short stories of decisions made with stated odds, learn whether each turned out well or badly, and rate the decision and the person's skill at such decisions.

5-7 minUntimed

Create a study with this test

Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.

Papers, 2021 to 2025
172
Evidence grade
C
Registered clinical studies
1
How these are counted
A new mill charged a third less, but 1 in 5 of its deliveries came a day late. The manager ordered from the new mill.
How it turned out: The flour came a day late.
The flour order
How good was this decision?
1
2
3
4
5
6
7
Outcome biasGood endings 5.2, bad 3.9
Read how a decision with stated odds turned out, then rate the decision
  • Overview
  • Evidence
  • Task design
  • Quality checks
  • Settings
  • Cite
  • Questions

What it measures

Measures the outcome bias. Participants read 12 short stories in which a person takes a chance with stated odds, such as ordering flour from a cheaper mill that is sometimes late, learn whether it turned out well or badly, and rate the decision and the person's skill at such decisions. The main score is how much better the same kind of decision is rated when it turned out well.

Constructs

  • Outcome bias
  • Decision evaluation
  • Separating luck from judgment
  • Judgment under risk

Research uses

  • Judgment and decision-making research
  • Management and organizational research
  • Debiasing and training research
  • Within-study group comparison

How the paradigm works

The decision quality task measures the outcome bias: people judge the same decision better when it turned out well than when it turned out badly, even when they know everything the decider knew. Baron and Hershey (1988) gave people descriptions of decisions made under stated risk, such as an operation with an 8% chance of death, together with their outcomes, and asked them to rate the quality of the thinking that went into each decision. In their first experiment, 20 students rated matched cases that differed only in success or failure; cases that succeeded were rated 0.70 points higher on a scale from -3 to 3, and only 9.3% of the pairs were rated higher after failure. The bias held in five experiments, for medical decisions and money gambles, for the decider's predicted competence as well as the decision (the two biases correlated 0.85 across people), and among people who said outcomes should not count. In a preregistered replication with 692 online participants who each saw one version, Aiyer and colleagues (2023) found decisions that succeeded rated 1.78 against 0.68 for those that failed on the same scale (d = 1.10 for a physician's decision and 0.77 for a patient's).

More about the paradigm

Here, 12 stories written for this task, such as a bakery ordering flour from a cheaper mill that is sometimes late, state the odds the decider faced and how the decision turned out; each story has a good and a bad ending that differ only in their last sentence. After each story, people rate how good the decision was and how good the person is at decisions like it, from 1 (very bad) to 7 (very good). As standard, each story comes once and half end well; a study can show each story twice, once with each ending, as in Baron and Hershey's within-person cases. The main score is the mean rating of the decisions that ended well minus that of those that ended badly.

Scores

Outcome Biasscore
Mean rating of how good the decision was for stories that ended well minus that for stories that ended badly, from -6 to 6.
Positive means the same kind of decision was judged better after a good outcome, the outcome bias
Outcome Bias on Competencescore
The same difference for the ratings of how good the person is at decisions like it.
Positive means the person was judged more skilled after a good outcome
Decision Rating After a Good Outcomescore
Mean rating of how good the decision was for stories that ended well, from 1 to 7.
Informational
Decision Rating After a Bad Outcomescore
Mean rating of how good the decision was for stories that ended badly, from 1 to 7.
Informational
Competence Rating After a Good Outcomescore
Mean rating of the person's skill at such decisions for stories that ended well, from 1 to 7.
Informational
Competence Rating After a Bad Outcomescore
Mean rating of the person's skill at such decisions for stories that ended badly, from 1 to 7.
Informational
Stories Rated Higher With the Good Ending%
With each story shown twice, the share of stories whose good ending got the higher decision rating; empty with each story once.
Informational
Median Reading Timems
Median time spent on a story screen before pressing Next.
Informational
Median Rating Timems
Median time from a rating screen appearing to a rating, over ratings at or after 100 ms.
Informational
Ratings Under 100 mscount
Ratings made less than 100 ms after their screen appeared.
0 expected
Stories Ratedcount
Story screens rated on both questions.
Informational

Use and evidence

172 papers from 2021 to 2025 name this task in their title or abstract, up 59% compared with 2016 to 2020. That ranks it 192 of 711 test families in our census. See the counts for every test.

2021 to 2025, the years gradedEarlier years
Papers per year naming the Decision Quality Task, 2000 to 2025Bar chart. 2021: 26; 2022: 33; 2023: 35; 2024: 33; 2025: 45 papers. Peak year 2025.02040602000: 3 papers2001: 4 papers2002: 4 papers2003: 5 papers2004: 8 papers2005: 10 papers2006: 10 papers2007: 13 papers2008: 7 papers2009: 10 papers2010: 18 papers2011: 21 papers2012: 10 papers2013: 18 papers2014: 24 papers2015: 21 papers2016: 22 papers2017: 15 papers2018: 24 papers2019: 22 papers2020: 25 papers2021: 26 papers2022: 33 papers2023: 35 papers2024: 33 papers2025: 45 papers200020052010201520202025
Papers per year naming the Decision Quality Task, 2000 to 2025Bar chart. 2021: 26; 2022: 33; 2023: 35; 2024: 33; 2025: 45 papers. Peak year 2025.02040602000: 3 papers2001: 4 papers2002: 4 papers2003: 5 papers2004: 8 papers2005: 10 papers2006: 10 papers2007: 13 papers2008: 7 papers2009: 10 papers2010: 18 papers2011: 21 papers2012: 10 papers2013: 18 papers2014: 24 papers2015: 21 papers2016: 22 papers2017: 15 papers2018: 24 papers2019: 22 papers2020: 25 papers2021: 26 papers2022: 33 papers2023: 35 papers2024: 33 papers2025: 45 papers200020052010201520202025
Papers per year whose title or abstract names this task (OpenAlex).
Show the numbers
YearPapers
202545
202433
202335
202233
202126
202025
201922
201824
201715
201622
201521
201424
201318
201210
201121
201018
200910
20087
200713
200610
200510
20048
20035
20024
20014
20003
C

Use grade C

100 to 299 papers from 2021 to 2025.

C

Evidence grade C

At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.

Papers on test-retest reliability
0
Papers on reliability or validity
43
Registered clinical studies using it as an outcome
1

Measured on ConductCognition

Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.

No pooled figure yet.

Note on the counts: includes clinical and management papers that name the bias.

How the grades work

Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.

Source: ConductCognition test census 0.58.0, counted 13 November 2026 from OpenAlex and ClinicalTrials.gov.

Task design

The trial sequence as the test runs in ConductCognition.

Stories
12 written for this task, each a person taking a chance with stated odds, such as a bakery ordering flour from a cheaper mill that is late 1 time in 5, with the safer choice it gives up. Each has a good and a bad ending that differ only in their last sentence; the chances run from 1 in 10 to 2 in 5. None has political, religious, group, health or injury content, and none is a published case. They are shown in the session's language.
Ratings
After each story, two screens: how good the decision was and how good the person is at decisions like it, from 1 (very bad) to 7 (very good) on seven buttons. A check with feedback first makes sure people know each person had only the information in the story.
Order
12 stories as standard, or the first 8. Each story comes once, half of them ending well and half badly as drawn from the session's seed, in an order drawn from it too. A study can show each story twice, once with each ending, one ending in each half of the session, as in Baron and Hershey's within-person cases.
Scores
The outcome bias: the mean decision rating of stories that ended well minus that of stories that ended badly, from -6 to 6. Also the same difference for the ratings of the person's skill, the four means behind them and, with each story twice, the share of stories rated higher with the good ending.

Baron and Hershey (1988) told people everything the decider knew and still found decisions rated better after good outcomes, and Aiyer and colleagues (2023) replicated it with 692 participants. This task uses their method with stories of its own, none from either source. Scores describe how much outcomes move ratings of decisions and are best compared between groups; they do not show how well a person decides and do not diagnose.

Quality checks

Each session is marked valid, needs review or invalid, with the reason, before you see its scores.

Scored rows: the scored run's ratings of the stories. No rating is correct, so accuracy is not judged against chance. A run with no rated story that ended well or none that ended badly, or with fewer than half the stories rated on both questions, is invalid, and any other shortfall is flagged; one number chosen for at least 90% of the ratings is flagged. Ratings under 200 ms on at least half the screens invalidate.

Why these rules

There is no correct option to compare with chance, so accuracy is not judged against chance. The scores describe how much an outcome moves the rating of a decision made with the same information, so any rating is a possible answer, and so is rating every decision the same. The outcome bias needs stories with both endings rated. One number for nearly every rating suggests the scale was not used; it is flagged for review because it can also be a real answer. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.

No percentile. Scores and quality flags only.

Running this test

Settings you can change

SettingStandardAllowed
Stories12 stories (standard)8 stories, 12 stories (standard)
EndingsEach story once, with one ending (standard)Each story once, with one ending (standard), Each story twice, once with each ending

Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.

How test settings work

Need a setting this test does not have?

Devices and languages

  • Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
  • Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
  • Available on: Every plan, including the free plan.
  • No percentile. Scores and quality flags only. How norms work
  • Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.

Cite and describe

Methods text, with the standard settings

Participants completed the Decision Quality Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Baron J, Hershey JC, 1988; Aiyer S, Kam HC, Ng KY, Young NA, Shi J, Feldman G, 2023), with the standard settings (stories: 12 stories; endings: Each story once, with one ending). The main score was outcome bias (mean rating of how good the decision was for stories that ended well minus that for stories that ended badly, from -6 to 6).

ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.

  • Baron, J., & Hershey, J. C. (1988). Outcome bias in decision evaluation. Journal of Personality and Social Psychology, 54(4), 569-579. https://doi.org/10.1037/0022-3514.54.4.569
  • Aiyer, S., Kam, H. C., Ng, K. Y., Young, N. A., Shi, J., & Feldman, G. (2023). Outcomes affect evaluations of decision quality: Replication and extensions of Baron and Hershey's (1988) outcome bias Experiment 1. International Review of Social Psychology, 36(1), 12. https://doi.org/10.5334/irsp.751A preregistered replication with 692 participants, each rating one version of one decision.

Source papers

  • Aiyer and colleagues (2023): 692 adults in the United States, recruited online, each rated one version of one medical decision from -3 to 3; decisions that succeeded were rated 1.78 (SD 0.81) and those that failed 0.68 (SD 1.47).Paradigm source, Adults

Questions

How long does the Decision Quality Task take?

About 5 to 7 minutes, as listed in the test library.

What device does a participant need?

Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.

What scores does it produce?

Outcome Bias, Outcome Bias on Competence, Decision Rating After a Good Outcome, Decision Rating After a Bad Outcome, Competence Rating After a Good Outcome, Competence Rating After a Bad Outcome, Stories Rated Higher With the Good Ending, Median Reading Time, Median Rating Time, Ratings Under 100 ms and Stories Rated. Each session is also marked valid, needs review or invalid, with the reason.

How widely is the Decision Quality Task used in research?

172 papers from 2021 to 2025 name it in their title or abstract, up 59% compared with 2016 to 2020. That ranks it 192 of 711 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.58.0, counted 13 November 2026 from OpenAlex and ClinicalTrials.gov.

How much reliability evidence does the Decision Quality Task have?

In the published research, 0 papers report its test-retest reliability and 1 registered clinical study uses it as an outcome (evidence grade C in the ConductCognition test census).

Does it report a percentile?

No. Scores and quality flags only.

Which settings can a researcher change?

Stories (8 stories, 12 stories (standard)) and endings (Each story once, with one ending (standard), Each story twice, once with each ending). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.

Which languages can participants use?

Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.

Is this a clinical test?

No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.

Every test: the full test list.

New tests and features, about once a month. Unsubscribe at any time.

Run this test in your next study

Start free with 10 free participants on any test, then use participant credits or a yearly plan.

Start free

Related tests

Hindsight Bias TaskReasoning Biases ProblemsRisky Framing Task
ConductCognition

Hosted browser-native cognitive testing for academic and commercial research workflows.

A product of ConductScience, Inc.

Tests

  • All cognitive tests
  • Test batteries
  • Use and evidence
  • Test norms
  • Trail Making Test
  • N-Back Task
  • Big Five personality test
  • Automated scoring
  • Study design
  • New tests by email

Guides

  • Methods and questions
  • Blog
  • Recruitment setup
  • Participant devices
  • Prolific
  • Sona Systems
  • CloudResearch Connect
  • Qualtrics

Use cases

  • Student participant pools
  • Aging and dementia research
  • Clinical trials
  • US clinical research
  • Remote testing

Compare and buy

  • Pricing
  • Compare platforms
  • Offline edition
  • FAQ

Trust

  • Study Quality Monitor
  • HIPAA-ready testing
  • Data export
  • Privacy Policy
  • Terms of Use
  • Data Policy

Company

  • ConductScience
  • ConductScience Fellows
© 2026 ConductScience, Inc. Research use only.