Decision Making
Four-Armed Bandit Task
Also called: multi-armed bandit, restless bandit, explore-exploit task
Measure exploration and exploitation with four machines whose payoffs drift over time, scored by points won and best-option choices.
11-15 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 357
- Evidence grade
- B
- Registered clinical studies
- 15
What it measures
Measures how people balance exploring and exploiting: they choose among four machines whose payoffs drift slowly and independently.
Constructs
- Exploration and exploitation
- Reward learning
- Decision making under uncertainty
- Behavioral flexibility
Research uses
- Reinforcement-learning and computational psychiatry research
- Decision-making studies
- Development and aging research
- Within-study group comparison
How the paradigm works
The Four-Armed Bandit measures how people balance exploring and exploiting. On each trial, participants choose one of four machines and win the points it pays, between 1 and 100.
More about the paradigm
The task follows Daw and colleagues (2006). Each machine's mean payoff drifts by their decaying Gaussian random walk: on every trial it moves 1.64% of the way back toward 50 and takes a random step with a standard deviation of 2.8 points, separately for each machine. A machine pays its current mean plus noise with a standard deviation of 4 points. Because the best machine changes over the task, participants must keep trying machines to track it. The walks come from the session's seed, and each trial saves all four means and payoffs. The standard task has 300 trials in two halves with a break, after 5 practice trials, and allows 1.5 seconds per choice; a missed choice earns no points.
Scores
- Points Share of Maximum%
- Points won divided by the points the best-paying machine paid on the same trials.
- Higher is better
- Best Option Rate%
- Share of choices of the machine with the highest mean payoff at that moment.
- Higher is better
- Switch Rate%
- Share of choices that changed machine from the trial before.
- Informational
- Win-Stay Rate%
- After a payoff at least as high as the one before, the share of choices that stayed with the same machine.
- Informational
- Lose-Shift Rate%
- After a lower payoff than the one before, the share of choices that switched machine.
- Informational
- Total Pointspoints
- Points won over the test trials.
- Higher is better
- Missed Trialscount
- Trials with no choice within the time limit.
- Lower is better
- Median Choice Timems
- Median time to choose on answered trials.
- Informational
Use and evidence
357 papers from 2021 to 2025 name this task in their title or abstract, up 114% compared with 2016 to 2020. That ranks it 71 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 105 |
| 2024 | 89 |
| 2023 | 56 |
| 2022 | 49 |
| 2021 | 58 |
| 2020 | 42 |
| 2019 | 37 |
| 2018 | 41 |
| 2017 | 27 |
| 2016 | 20 |
| 2015 | 29 |
| 2014 | 19 |
| 2013 | 10 |
| 2012 | 20 |
| 2011 | 17 |
| 2010 | 14 |
| 2009 | 13 |
| 2008 | 15 |
| 2007 | 20 |
| 2006 | 9 |
| 2005 | 2 |
| 2004 | 4 |
| 2003 | 3 |
| 2002 | 4 |
| 2001 | 1 |
| 2000 | 2 |
Use grade B
300 to 999 papers from 2021 to 2025.
Evidence grade B
At least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers.
- Papers on test-retest reliability
- 9
- Papers on reliability or validity
- 39
- Registered clinical studies using it as an outcome
- 15
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Machines
- Four machines in fixed places, colored and numbered 1 to 4. Click or tap one, or press its number.
- Payoffs
- A machine pays its current mean plus Gaussian noise (SD 4), rounded and kept between 1 and 100 points.
- Walks
- Each mean decays by 0.9836 toward 50 and takes a Gaussian step (SD 2.8) every trial, generated from the session's seed.
- Trials
- 300 trials standard (100 to 300), with a break halfway, after 5 practice trials.
- Time to choose
- 1,500 ms standard, 3,000 ms or no limit. A missed choice shows a red X and earns no points.
- Feedback
- The points won show on the chosen machine for 1,000 ms, then a 500 ms gap before the next trial.
- Money
- Points are hypothetical by default; a setting changes the instruction line to say the study pays.
Daw and colleagues showed each participant one of three fixed walks in a scanner, with longer trials. Here every session gets its own walks by the same rule, and trials are paced for behavioral testing. The paper's reinforcement-learning model fits are not computed.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: the scored run's test choices. No machine is correct in advance, so accuracy is not judged against chance. Choosing one machine on 90% of choices is flagged, not invalid. Missing more than 25% of choices invalidates and more than 5% is flagged. Choices under 100 ms on at least half the trials invalidate; any is flagged.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. Staying with a machine that keeps paying well is a valid strategy, so a repeated machine is only flagged. Daw et al. (2006) report one or two missed choices in 300 at a 1.5 second limit, so missing more than a quarter means the participant was not doing the task, and more than 1 in 20 is unusual. Choices come quickly in this task (Daw et al. report a mean near 430 ms), so only a choice under 100 ms, which cannot follow from the screen, counts as too fast; at least half of choices that fast invalidates the session.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Number of trials | 300 | 100 to 300 |
| Time to choose | 1.5 seconds (standard) | 1.5 seconds (standard), 3 seconds, No limit |
| Payment line | Hypothetical points (standard) | Hypothetical points (standard), The study pays for points |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Four-Armed Bandit task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ, 2006), with the standard settings (number of trials: 300; time to choose: 1.5 seconds; payment line: Hypothetical points). The main score was points share of maximum (points won divided by the points the best-paying machine paid on the same trials).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876-879. https://doi.org/10.1038/nature04766
Source papers
- Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095):876-879.Paradigm source, Adults
Questions
How long does the Four-Armed Bandit Task take?
About 11 to 15 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Points Share of Maximum, Best Option Rate, Switch Rate, Win-Stay Rate, Lose-Shift Rate, Total Points, Missed Trials and Median Choice Time. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Four-Armed Bandit Task used in research?
357 papers from 2021 to 2025 name it in their title or abstract, up 114% compared with 2016 to 2020. That ranks it 71 of 395 test families in the ConductCognition test census (use grade B). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Four-Armed Bandit Task have?
In the published research, 9 papers report its test-retest reliability and 15 registered clinical studies use it as an outcome (evidence grade B in the ConductCognition test census).
Which settings can a researcher change?
Number of trials (100 to 300), time to choose (1.5 seconds (standard), 3 seconds, No limit) and payment line (Hypothetical points (standard), The study pays for points). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.