Executive Function
Decisions from Experience Task
Also called: sampling paradigm, description-experience gap task, experience-based choice task, free sampling task, decisions from description and experience
Measure how much rare outcomes count when people learn two options by opening them rather than reading their stated chances, scored as the description-experience gap.
6-9 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 115
- Evidence grade
- D
- Registered clinical studies
- 0
What it measures
Measures how much rare outcomes count in choices learned by experience, compared with choices made from stated chances. In 6 problems participants open two boxes as often as they like to learn the points they give, then choose one; in 6 others each box's outcomes and chances are stated. The main score is the description-experience gap: how much more often choices act as if rare outcomes did not count after opening the boxes than after reading about them.
Constructs
- Decision making under risk
- Learning from experience
- Weighting of rare outcomes
- Information search
Research uses
- Research on choices under risk and the description-experience gap
- Decision-making and judgment research
- Within-study comparison of learning by sampling and reading stated chances
How the paradigm works
The decisions from experience task compares choices whose outcomes are learned by sampling with choices whose outcomes and chances are stated. When people read two options' outcomes and chances (decisions from description), they choose as if rare outcomes counted more than their chances warrant, as prospect theory describes; when they learn the options by sampling outcomes one at a time (decisions from experience), they choose as if rare outcomes counted less (Hertwig, Barron, Weber and Erev, 2004). In Hertwig and colleagues' sampling paradigm, people clicked either of two buttons as often as they wished, each click drawing one outcome with replacement, then chose one option for a real payoff; the median was 15 draws per problem, and 18 of the 25 people who sampled an option with a 10% chance of 32 never saw the 32. Across studies with the sampling paradigm the gap averaged 18.7 percentage points for choices between a risky and a safe option and 7 points between two risky options (Wulff, Mergenthaler-Canseco and Hertwig, 2018, as Hertwig and Wulff, 2022, report). Prospect theory applied to the outcomes people actually saw fits their choices about as well as it fits described ones (Fox and Hadar, 2006), and underweighting was also found when samples were representative (Ungemach, Chater and Stewart, 2009).
More about the paradigm
Each problem shows two new boxes that hold points: a safe box that gives the same points every time, and a risky box that gives one outcome rarely (a 5%, 10% or 20% chance) and another otherwise, one of the two being 0. In rare-good problems the rare outcome is the larger; in rare-bad problems it is the 0. The 12 problems are this test's own, in six pairs of twins with the same chance and kind; the session's seed sends one twin of each pair to each condition. First come 6 sampled problems: the participant opens either box as often as they like, each opening showing the points of one draw, with replacement, from draws set by the seed; each box must be opened once, at most 100 openings are allowed, and openings do not count. Then 6 described problems state each box's outcomes and chances. In every problem one box is chosen and one draw from it counts; the total is shown at the end. A check question with feedback and a practice problem come first. About 6 to 9 minutes.
The main score is the description-experience gap: the share of sampled problems chosen as if the rare outcome did not count (the risky box in rare-bad problems, the safe box in rare-good ones), minus the same share for described problems. Positive values mean rare outcomes counted less after sampling. The two shares, the gap within rare-good and rare-bad problems, the share of sampled problems in which the rare outcome never appeared, the median number of openings and the share of choices of the box with the higher mean in the openings seen are reported too. Twelve choices make one person's gap coarse, so compare groups or conditions. Scores support research and do not diagnose.
Scores
- Description-Experience Gap%
- Share of sampled problems chosen as if the rare outcome did not count, minus the same share for described problems.
- Positive: rare outcomes counted less after sampling; 0 is no gap
- Rare Outcome Discounted, Sampled%
- Share of sampled problems chosen as if the rare outcome did not count: the risky box when the rare outcome is 0, the safe box when it is the larger.
- Informational
- Rare Outcome Discounted, Described%
- The same share for described problems.
- Informational
- Gap, Rare Large Outcome%
- In problems whose rare outcome is the larger, the share of safe choices when sampled minus when described.
- Positive: the rare large outcome counted less after sampling
- Gap, Rare Zero Outcome%
- In problems whose rare outcome is 0, the share of risky choices when sampled minus when described.
- Positive: the rare 0 counted less after sampling
- Rare Outcome Never Seen%
- Share of sampled problems in which the risky box never showed its rare outcome.
- Informational
- Median Openingscount
- Median number of box openings per sampled problem, both boxes together.
- 2 or fewer is flagged for review
- Higher Sample Mean Chosen%
- Share of sampled problems in which the box chosen had the higher mean over the openings seen; ties left out.
- Informational
- Points Wonpoints
- Points of the draws that counted, one from each box chosen.
- Informational
- Check Correctflag
- 1 when the check question was answered correctly, 0 when not.
- 1 supports interpretation
- Median Choice Timems
- Median time from the choice screen to the choice, at or after 100 ms.
- Informational
- Anticipationscount
- Presses within 100 ms of a screen or of the last opening. They are not taken.
- Quality check
- Problems Answeredcount
- Problems answered in the scored run, both conditions.
- Fewer than planned is flagged for review
- Planned Problemscount
- Problems set for the session.
- Informational
Use and evidence
115 papers from 2021 to 2025 name this task in their title or abstract, down 2% compared with 2016 to 2020. That ranks it 200 of 495 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 23 |
| 2024 | 18 |
| 2023 | 19 |
| 2022 | 25 |
| 2021 | 30 |
| 2020 | 16 |
| 2019 | 19 |
| 2018 | 28 |
| 2017 | 37 |
| 2016 | 17 |
| 2015 | 28 |
| 2014 | 29 |
| 2013 | 18 |
| 2012 | 23 |
| 2011 | 18 |
| 2010 | 12 |
| 2009 | 17 |
| 2008 | 8 |
| 2007 | 8 |
| 2006 | 4 |
| 2005 | 0 |
| 2004 | 1 |
| 2003 | 0 |
| 2002 | 1 |
| 2001 | 0 |
| 2000 | 0 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 1
- Papers on reliability or validity
- 14
- Registered clinical studies using it as an outcome
- 0
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.22.0, counted 3 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Problems
- 12 problems of this test's own, in six pairs of twins: a safe box that gives the same points every time against a risky box whose rare outcome has a 5%, 10% or 20% chance, the larger outcome in three pairs and 0 points in the other three. The risky box's expected points are 1.05 to 1.13 times the safe box's. The session's seed sends one twin of each pair to each condition and sets the order and the sides.
- Sampled problems
- 6, first. The participant opens either box by click or tap, as often as they like, each opening showing the points of one draw with replacement; each box must be opened at least 1 time, and at most 100 openings are allowed. Openings do not count. The participant then presses Choose a box and picks one.
- Described problems
- 6, after the sampled ones. Each box's outcomes and their chances are stated, and one box is chosen.
- Points
- Each chosen box is opened once more at the end, and those points count. A check question with feedback and a practice problem come first. The payment line says that the points are not exchanged for money, or gives the paid line when the study pays for points.
- Scores
- The description-experience gap (the share of sampled problems chosen as if the rare outcome did not count, minus the same share for described problems), the two shares, the gap within each kind of problem, the share of sampled problems whose rare outcome never appeared, and the median number of openings.
Hertwig and colleagues (2004) compared two groups of people, each answering three of six problems for money. This test gives each person both kinds of problems, with problems of its own and points that are not money unless the study pays for them, so its gaps are not directly comparable with published ones. The sampled problems always come first, so that no stated chance is read before the boxes are opened. Twelve choices make one person's gap coarse, so compare groups or conditions. The scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: decisions_from_experience_sampled and decisions_from_experience_described from the scored run, one choice of a box per problem with no time limit. No box is the right answer, so accuracy is not judged against chance. A run short of its planned problems, a wrong check answer and a median of 2 or fewer openings per sampled problem are flagged for review. Choices under 200 ms on at least half the problems invalidate.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. The two boxes of every problem are close in expected points, so either choice is reasonable. Opening each box only once makes a sampled choice rest on one outcome, which the test allows and flags. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Payment line | Hypothetical points (standard) | Hypothetical points (standard), The study pays for points |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Decisions from Experience Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Hertwig, Barron, Weber & Erev, 2004; Wulff, Mergenthaler-Canseco & Hertwig, 2018; Hertwig & Wulff, 2022), with the standard settings (payment line: Hypothetical points). The main score was description-experience gap (share of sampled problems chosen as if the rare outcome did not count, minus the same share for described problems).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534-539. https://doi.org/10.1111/j.0956-7976.2004.00715.xThe design: two buttons, each drawing an outcome with replacement as often as the person wished, then one choice for a real payoff, against the same problems described; rare events counted less after sampling.
- Fox, C. R., & Hadar, L. (2006). "Decisions from experience" = sampling error + prospect theory: Reconsidering Hertwig, Barron, Weber & Erev (2004). Judgment and Decision Making, 1(2), 159-161. https://doi.org/10.1017/S1930297500002370Prospect theory applied to the outcomes people actually sampled fitted their choices about as well as it fits described ones.
- Ungemach, C., Chater, N., & Stewart, N. (2009). Are probabilities overweighted or underweighted when rare outcomes are experienced (rarely)? Psychological Science, 20(4), 473-479. https://doi.org/10.1111/j.1467-9280.2009.02319.xUnderweighting of rare outcomes when the samples people saw were representative of the chances.
- Wulff, D. U., Mergenthaler-Canseco, M., & Hertwig, R. (2018). A meta-analytic review of two modes of learning and the description-experience gap. Psychological Bulletin, 144(2), 140-176. https://doi.org/10.1037/bul0000115The gap counted as the difference in choices consistent with underweighting rare events, larger for a risky against a safe option than for two risky options.
- Hertwig, R., & Wulff, D. U. (2022). A description-experience framework of the psychology of risk. Perspectives on Psychological Science, 17(3), 631-651. https://doi.org/10.1177/17456916211026896The meta-analysis's gap sizes for the sampling paradigm: 18.7 percentage points for a risky against a safe option, 7 for two risky options.
Source papers
- Hertwig R, Barron G, Weber EU, Erev I (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8):534-539.N 100, University students
- No normative data for this task.Paradigm source, N/A
Questions
How long does the Decisions from Experience Task take?
About 6 to 9 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Description-Experience Gap, Rare Outcome Discounted, Sampled, Rare Outcome Discounted, Described, Gap, Rare Large Outcome, Gap, Rare Zero Outcome, Rare Outcome Never Seen, Median Openings, Higher Sample Mean Chosen, Points Won, Check Correct, Median Choice Time, Anticipations, Problems Answered and Planned Problems. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Decisions from Experience Task used in research?
115 papers from 2021 to 2025 name it in their title or abstract, down 2% compared with 2016 to 2020. That ranks it 200 of 495 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.22.0, counted 3 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Decisions from Experience Task have?
In the published research, 1 paper reports its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Payment line (Hypothetical points (standard), The study pays for points). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.