Decision Making
Best Offer Task
Also called: secretary problem, secretary task, optimal stopping task, best choice problem, full-information secretary problem, sultan's dowry problem
Measure when people stop searching: take or pass offers that arrive one at a time, and compare how often the best is taken and where people stop with the optimal rule.
4-7 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 16
- Evidence grade
- D
- Registered clinical studies
- 0
What it measures
Measures when people stop searching. Offers arrive one at a time, in random order, from a known number; each is taken or passed, and a passed offer cannot come back. Only the best offer of the set earns points. Scores how often the best offer is taken and where people stop compared with the optimal rule.
Constructs
- Optimal stopping
- Sequential decision making
- Search under uncertainty
- Threshold strategies
Research uses
- Judgment and decision-making research
- Behavioral economics of search
- Comparing search with ranks only and with values
How the paradigm works
The Best Offer Task measures when people stop searching. It is the secretary problem: offers from a known number arrive one at a time in random order, each is taken or passed, a passed offer cannot come back, and only the best of all the offers counts. Seale and Rapoport (1997) showed people only each offer's rank among those seen so far, and people stopped earlier than the optimal rule. Lee (2006) showed values from a known range.
More about the paradigm
This version uses our own numbers, wording and screens. Each set draws distinct whole numbers from 1 to 100 in a random order from the session's seed. As standard, each offer shows its rank among the offers so far, in a row that shows every rank seen; a setting shows each offer's value instead. Sets have 10 offers (5 to 40) and the session 15 sets (5 to 40). The last offer is taken when it is reached. Taking the best offer of a set earns 10 points, and a result screen after each set shows every offer. One check question with feedback comes first, and there is no time limit.
Scores
- Best Offer Rate%
- Share of completed sets in which the offer taken was the best of all the set's offers.
- Higher: the best offer taken more often; compare with the optimal rule's rate on the same sets
- Mean Stop Positioncount
- Mean position of the offer taken, from 1 to the number of offers.
- Informational
- Optimal Stop Positioncount
- Mean position where the optimal rule would have stopped on the same sets.
- Informational
- Early Stops%
- Share of sets stopped before the optimal rule would have stopped.
- Higher: stopping before the optimal rule, the common finding
- Late Stops%
- Share of sets stopped after the optimal rule would have stopped.
- Informational
- Estimated Cutoffcount
- Ranks only: the cutoff rule (pass the first offers, then take the first that is the best so far) that agrees with the most decisions; tied cutoffs are averaged.
- Below the optimal cutoff: a rule that starts taking too soon
- Optimal Cutoffcount
- Ranks only: the offer from which the optimal rule takes the first best-so-far offer, 4 of 10 as standard.
- Fixed for the number of offers
- Cutoff Fit%
- Ranks only: share of decisions the estimated cutoff rule matches; the forced last offer is left out.
- Near 1: a cutoff rule describes the choices well
- Optimal Rule Best Offer Rate%
- Share of the same sets that the optimal rule would have won.
- How favorable the session's sets were; informational
- Offers Taken That Were Not Best So Farcount
- Offers taken before the last that were not the best so far, which can never be the best of all.
- 0 expected
- Points Wonpoints
- 10 points for each set in which the best offer was taken.
- Informational
- Check Correctflag
- 1 for a correct answer to the check about which offer earns points, 0 for a wrong answer.
- A wrong answer is flagged for review
- Median Decision Timems
- Median time to take or pass an offer, over decisions at or after 100 ms; the forced last offer is left out.
- Informational
- Decisions Under 100 mscount
- Offers taken or passed less than 100 ms after they appeared.
- 0 expected
- Sets Completedcount
- Completed sets in the scored run.
- Fewer than planned is flagged for review
- Planned Setscount
- Sets set before the session.
- Informational
Use and evidence
16 papers from 2021 to 2025 name this task in their title or abstract, down 6% compared with 2016 to 2020. That ranks it 504 of 582 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 3 |
| 2024 | 3 |
| 2023 | 3 |
| 2022 | 5 |
| 2021 | 2 |
| 2020 | 2 |
| 2019 | 6 |
| 2018 | 3 |
| 2017 | 2 |
| 2016 | 4 |
| 2015 | 3 |
| 2014 | 4 |
| 2013 | 0 |
| 2012 | 1 |
| 2011 | 2 |
| 2010 | 0 |
| 2009 | 2 |
| 2008 | 1 |
| 2007 | 4 |
| 2006 | 6 |
| 2005 | 3 |
| 2004 | 4 |
| 2003 | 1 |
| 2002 | 0 |
| 2001 | 1 |
| 2000 | 2 |
Use grade E
Fewer than 30 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 1
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Sets
- 15 sets as standard, from 5 to 40, of 10 offers each as standard, from 5 to 40. Each set draws distinct whole numbers from 1 to 100 in a random order from the session's seed.
- Offers
- One at a time. The participant takes the offer or passes it; a passed offer cannot come back, and the last offer is taken when it is reached. As standard each offer shows its rank among the offers so far, in a row that updates every rank seen, as in Seale and Rapoport (1997). A setting shows each offer's value instead, after Lee (2006).
- Points
- Taking the best of all the offers in a set earns 10 points; any other offer earns nothing. A result screen after each set shows every offer. The study decides what points are worth, and the payment line says whether they are exchanged for money.
- Check
- One question with feedback before the first set: only the best of all the offers earns points, not the best so far.
- Optimal rule
- With ranks, pass the first offers and take the first that is the best so far, from offer 4 of 10, 8 of 20 and 16 of 40. With values, take a best-so-far offer when its chance of being the best is at least the chance of doing better by going on, worked out exactly for these draws.
- Scores
- The share of sets in which the best offer was taken; the mean stop position beside the optimal rule's on the same sets; the shares of sets stopped early and late; with ranks, the cutoff rule that agrees with the most decisions; and the optimal rule's own share of sets won. No time limit.
Seale and Rapoport (1997) showed ranks only, updated as each applicant arrived, and counted only the best applicant; Lee (2006) showed values between 0.00 and 100.00. This version uses its own numbers, with 10 offers and 15 sets as standard. Compare people with the same settings. Scores describe choices in the task, with no percentile or diagnosis.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: offers passed or taken in completed sets. Where a person stops is the measure, so no single decision is judged against chance. An unfinished run, a wrong check answer or the first offer taken in every one of at least 5 sets is flagged for review. Decisions under 200 ms on at least half the offers invalidate.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. Stopping early or late is what the task measures, and taking the first offer of every set wins one set in n by chance alone, so it is only flagged. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| What each offer shows | Its rank among the offers so far (standard) | Its rank among the offers so far (standard), Its value from 1 to 100 |
| Offers per set | 10 | 5 to 40 |
| Sets | 15 | 5 to 40 |
| Payment line | Hypothetical points (standard) | Hypothetical points (standard), The study pays for points |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Best Offer Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Seale DA, Rapoport A, 1997; Lee MD, 2006), with the standard settings (what each offer shows: Its rank among the offers so far; offers per set: 10; sets: 15; payment line: Hypothetical points). The main score was best offer rate (share of completed sets in which the offer taken was the best of all the set's offers).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Seale, D. A., & Rapoport, A. (1997). Sequential decision making with relative ranks: An experimental investigation of the "secretary problem". Organizational Behavior and Human Decision Processes, 69(3), 221-236. https://doi.org/10.1006/obhd.1997.2683The secretary problem with ranks only, and stopping earlier than the optimal rule.
- Lee, M. D. (2006). A hierarchical Bayesian model of human decision-making on an optimal stopping problem. Cognitive Science, 30(3), 555-580. https://doi.org/10.1207/s15516709cog0000_69The full-information version, with values from a known range.
- Goldstein, D. G., McAfee, R. P., Suri, S., & Wright, J. R. (2020). Learning when to stop searching. Management Science, 66(3), 1375-1394. https://doi.org/10.1287/mnsc.2018.3245An online version with values shown, in which stopping came close to the optimal rule with practice.
Source papers
Questions
How long does the Best Offer Task take?
About 4 to 7 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Best Offer Rate, Mean Stop Position, Optimal Stop Position, Early Stops, Late Stops, Estimated Cutoff, Optimal Cutoff, Cutoff Fit, Optimal Rule Best Offer Rate, Offers Taken That Were Not Best So Far, Points Won, Check Correct, Median Decision Time, Decisions Under 100 ms, Sets Completed and Planned Sets. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Best Offer Task used in research?
16 papers from 2021 to 2025 name it in their title or abstract, down 6% compared with 2016 to 2020. That ranks it 504 of 582 test families in the ConductCognition test census (use grade E). Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Best Offer Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
What each offer shows (Its rank among the offers so far (standard), Its value from 1 to 100), offers per set (5 to 40), sets (5 to 40) and payment line (Hypothetical points (standard), The study pays for points). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.
Related tests