Planning and Reasoning
Box Supply Task
Also called: beer game, beer distribution game, supply chain simulation, bullwhip effect, inventory management task
Measure stock decisions with shipping delays: manage one stage of a supply chain, track costs and see how order swings grow from shop to factory.
8-12 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 74
- Evidence grade
- D
- Registered clinical studies
- 0
What it measures
Measures stock-management decisions with shipping delays. Participants run one stage of a four-stage supply chain, filling orders and ordering boxes. Scores track costs and the growth of order swings along the chain.
Constructs
- Dynamic decision making
- Inventory management
- Feedback and delay
Research uses
- Behavioral operations research
- Research on planning with delayed consequences
How the paradigm works
The Box Supply Task measures stock-management decisions when actions have delayed effects. The participant runs one stage of a four-stage chain: shop, wholesaler, distributor or factory. The experiment described by Sterman (1989) showed how order fluctuations can grow as they pass along a supply chain.
More about the paradigm
This original adaptation uses boxes, points and three computer partners. The standard session is 32 rounds at the shop, with 20 to 60 rounds and any stage available. Every stage starts with 12 boxes and shipments in transit. Orders travel upstream in one round, shipments downstream in two, and factory production takes two rounds. Stock is received and orders are filled before the next order is chosen. Unfilled orders carry over. Participants see their own incoming orders, shipments, stock, backlog and all boxes still owed to them.
Customer demand starts at 4 for four rounds, then takes independent seeded values from 6 to 10. A held box costs 1 point per round and an unfilled box costs 2; the study can multiply both costs by 2 or 5. Partners use a fixed forecast and order-up-to rule, disclosed at the end. A worked cost check precedes play. Instructions, results and orders wait for the participant.
Scores
- Total Costpoints
- Sum of holding and backlog costs at the participant stage.
- Lower cost means less stock and backlog at these settings
- Participant Order Amplificationscore
- Participant order variance divided by customer-demand variance over completed rounds.
- Above 1 means larger order swings than customer demand
- Whole-Chain Costpoints
- Sum of holding and backlog costs across all four stages.
- Informational
- Shop Order Amplificationscore
- Variance of orders at this stage divided by customer-demand variance, including opening rounds.
- Above 1 means amplified order swings; null with constant demand
- Wholesaler Order Amplificationscore
- Variance of orders at this stage divided by customer-demand variance, including opening rounds.
- Above 1 means amplified order swings; null with constant demand
- Distributor Order Amplificationscore
- Variance of orders at this stage divided by customer-demand variance, including opening rounds.
- Above 1 means amplified order swings; null with constant demand
- Factory Order Amplificationscore
- Variance of orders at this stage divided by customer-demand variance, including opening rounds.
- Above 1 means amplified order swings; null with constant demand
- Mean Stockcount
- Average stock left at the participant stage after filling orders.
- Informational
- Mean Unfilled Orderscount
- Average unfilled boxes at the participant stage after shipping.
- Informational
- Mean Ordercount
- Average boxes ordered by the participant.
- Informational
- Customer Demand Variancescore
- Population variance of customer demand over the scored rounds.
- Denominator of the order-amplification ratios
- Check Correctflag
- 1 for a correct cost check, 0 for an incorrect answer.
- A wrong answer is flagged for review
- Median Choice Timems
- Median confirmation time for choices at or after 100 ms.
- Informational
- Choices Under 100 mscount
- Confirmed orders made less than 100 ms after the screen appeared.
- 0 expected
- Rounds Completedcount
- Length of the consecutive readable order record from round 1 in the selected run.
- Fewer than planned is flagged for review
- Planned Roundscount
- Number of rounds set before the session.
- Informational
Use and evidence
74 papers from 2021 to 2025 name this task in their title or abstract, down 15% compared with 2016 to 2020. That ranks it 290 of 582 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 17 |
| 2024 | 12 |
| 2023 | 8 |
| 2022 | 18 |
| 2021 | 19 |
| 2020 | 18 |
| 2019 | 14 |
| 2018 | 18 |
| 2017 | 15 |
| 2016 | 22 |
| 2015 | 22 |
| 2014 | 33 |
| 2013 | 22 |
| 2012 | 19 |
| 2011 | 16 |
| 2010 | 19 |
| 2009 | 23 |
| 2008 | 22 |
| 2007 | 14 |
| 2006 | 24 |
| 2005 | 15 |
| 2004 | 11 |
| 2003 | 11 |
| 2002 | 8 |
| 2001 | 2 |
| 2000 | 6 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 15
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes teaching uses and simulation models of the game.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.39.0, counted 10 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Stages and rounds
- Run the shop as standard, or select wholesaler, distributor or factory. The other three stages are simulated partners. Standard length is 32 rounds, with 20 to 60 available. Orders range from 0 to 100 boxes.
- Delays and stock
- Each stage starts with 12 boxes, no backlog and shipments of 4 due in each of the next two rounds. Orders reach the next stage after one round; shipments take two rounds. Factory production takes two rounds. Stock is received, orders are filled, costs are assessed, then new orders are placed. Unfilled orders carry forward.
- Demand and points
- Customer demand is 4 in each of the first four rounds, then a seeded independent integer from 6 to 10 inclusive. Each round costs 1 point per held box and 2 per backordered box. The study may multiply both costs by 2 or 5. There is no monetary payment.
- Partners
- The fixed partner rule is disclosed at the end. Forecast demand is 75% of the previous forecast plus 25% of incoming orders, starting at 4. Target inventory position is 12 plus 3 times forecast demand, or 2 times at the factory. Orders are target minus stock plus backlog minus all outstanding orders, rounded and bounded at 0 to 100.
- Scores
- Total participant cost is primary. Bullwhip is order variance divided by customer-demand variance over all completed rounds, separately for each stage and the participant. Ratios above 1 indicate amplified swings. A zero denominator gives no ratio. Chain cost, mean stock and mean backlog describe the same run. No lag shift or opening-round exclusion is applied.
This is our numeric adaptation of the four-stage stock-management experiment, with shorter order delays, seeded demand and fixed computer partners. Compare people with the same stage, round count and costs. Scores describe decisions in this task, with no percentile or diagnostic output.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: consecutive confirmed orders from round 1. An unfinished run or wrong check answer is flagged for review. Choices under 200 ms on at least half the rounds invalidate.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. Repeating an order can be a deliberate inventory policy, so constant quantities are accepted. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Rounds | 32 | 20 to 60 |
| Participant stage | Shop | Shop, Wholesaler, Distributor, Factory |
| Point cost multiplier | 1 | 1, 2, 5 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Box Supply Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Sterman JD, 1989), with the standard settings (rounds: 32; participant stage: Shop; point cost multiplier: 1). The main score was total cost (sum of holding and backlog costs at the participant stage).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Sterman, J. D. (1989). Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic Decision Making Experiment. Management Science, 35(3), 321-339. https://doi.org/10.1287/mnsc.35.3.321The four-stage stock-management experiment, shipment delays and growth of order fluctuations along the chain.
Source papers
Questions
How long does the Box Supply Task take?
About 8 to 12 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Total Cost, Participant Order Amplification, Whole-Chain Cost, Shop Order Amplification, Wholesaler Order Amplification, Distributor Order Amplification, Factory Order Amplification, Mean Stock, Mean Unfilled Orders, Mean Order, Customer Demand Variance, Check Correct, Median Choice Time, Choices Under 100 ms, Rounds Completed and Planned Rounds. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Box Supply Task used in research?
74 papers from 2021 to 2025 name it in their title or abstract, down 15% compared with 2016 to 2020. That ranks it 290 of 582 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.39.0, counted 10 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Box Supply Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Rounds (20 to 60), participant stage (Shop, Wholesaler, Distributor, Factory) and point cost multiplier (1, 2, 5). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.