Social Cognition
Reference Game
Also called: referential communication game, pragmatic reference task, rational speech act task
Measure how people use context to understand a message. Participants choose an object from one word and choose a word for a simulated listener.
5-7 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 81
- Evidence grade
- D
- Registered clinical studies
- 0
“blue”
Which word would help the listener?
blue · circleWhat it measures
Measures how people use context to understand and choose a one-word message. Participants select an object named by a simulated speaker and choose a word for a simulated listener. Scores separate context-sensitive choices, clear messages and informative descriptions.
Constructs
- Pragmatic inference
- Referential communication
- Message informativeness
Research uses
- Social cognition research
- Pragmatic language research
- Within-study group comparison
How the paradigm works
The Reference Game measures how people use the other objects in view to interpret a one-word message. Participants choose an object for a simulated speaker and choose a true word to help a simulated listener identify a marked object. All three objects are visible to both players.
More about the paradigm
Each role has 24 choices by default, or 12 as a setting. Equal numbers of unique, asymmetric overlapping and symmetric contexts separate clear reference from contextual inference. The role order, colors, shapes, object positions and word positions follow the saved seed. One practice choice introduces each role, with feedback; test choices receive no feedback.
The primary measure is the percentage of ambiguous asymmetric listener choices that select the object favored by an informative-speaker model. Its word weights are the reciprocal of the number of objects matching each true word, with a uniform object prior. This is a model-consistent choice, not an objectively correct answer. Informative-message choices compare the two true words only when one fits fewer objects. Unique-message accuracy and literal consistency provide separate comprehension checks.
Scores
- Context-Sensitive Choices%
- Model-consistent choices among answered asymmetric ambiguous listener trials.
- Descriptive research measure
- Informative Messages%
- Choices of the word matching fewer objects, when the two true words differ in specificity.
- Descriptive research measure
- Unique-Message Accuracy%
- Correct referents among answered unique listener messages.
- Descriptive research measure
- Literal Consistency%
- Listener choices that match the given word.
- Descriptive research measure
- Median Response Timems
- Median answered-choice time, excluding answers under 100 ms.
- Descriptive research measure
- Choices Answeredcount
- Recognized object and word choices.
- Descriptive research measure
- Choices Setcount
- Planned choices in both roles.
- Descriptive research measure
- Answer Rate%
- Answered choices as a percentage of those planned.
- Descriptive research measure
- Anticipationscount
- Answers under 100 ms.
- Descriptive research measure
Use and evidence
81 papers from 2021 to 2025 name this task in their title or abstract, up 45% compared with 2016 to 2020. That ranks it 273 of 582 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 30 |
| 2024 | 19 |
| 2023 | 7 |
| 2022 | 9 |
| 2021 | 16 |
| 2020 | 18 |
| 2019 | 14 |
| 2018 | 9 |
| 2017 | 6 |
| 2016 | 9 |
| 2015 | 2 |
| 2014 | 2 |
| 2013 | 3 |
| 2012 | 0 |
| 2011 | 0 |
| 2010 | 0 |
| 2009 | 1 |
| 2008 | 2 |
| 2007 | 2 |
| 2006 | 0 |
| 2005 | 2 |
| 2004 | 0 |
| 2003 | 2 |
| 2002 | 0 |
| 2001 | 0 |
| 2000 | 0 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 7
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes games between two people and studies of language models.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Display
- 3 generated objects vary in color and shape. Both players see the same objects; the partner is simulated.
- Session
- 24 choices per role by default, or 12. Each role contains equal numbers of unique, asymmetric overlapping and symmetric contexts.
- Roles
- As listener, choose an object from a one-word message; as speaker, choose one of two true words for a marked object. Role order and displays follow the seed.
- Practice
- 1 separate practice choice per role, with feedback. Test choices receive no feedback.
- Timing
- No response deadline. A 500 ms lead-in precedes each role, with a 300 ms gap after choices.
- Languages
- Instructions and messages follow the session language in English, Italian, German, Spanish, French, Japanese, Portuguese and Chinese.
A repeated forced-choice adaptation of the published probability-allocation task. Ambiguous listener choices are compared with an informative-speaker model with a uniform prior. Scores describe this task; percentiles are off.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: reference_game_choice from the most complete run. Ambiguous choices are descriptive, not errors. Fewer than 75% of planned choices answered invalidates; other incomplete runs and fewer than 50% correct unique messages are flagged. Answers under 200 ms on at least half of 4 or more answers invalidate. A hidden tab is flagged.
Why these rules
Context-sensitive choices and message preferences are the outcomes. Unique messages check comprehension separately. Completion and fast-response limits are ConductScience choices; reading times are not pure reaction times.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Choices in each role | 24 choices (standard) | 12 choices, 24 choices (standard) |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Reference Game in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Frank MC, Goodman ND, 2012), with the standard settings (choices in each role: 24 choices). The main score was context-sensitive choices (model-consistent choices among answered asymmetric ambiguous listener trials).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Frank, M. C., & Goodman, N. D. (2012). Predicting pragmatic reasoning in language games. Science, 336(6084), 998. https://doi.org/10.1126/science.1218633One-word reference and informative-speaker reasoning; our displays, repeated choices and uniform prior are adaptations.
Source papers
- Frank, M. C., & Goodman, N. D. (2012). Predicting pragmatic reasoning in language games. Science, 336(6084), 998.Paradigm source, Adults in the source study
Questions
How long does the Reference Game take?
About 5 to 7 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Context-Sensitive Choices, Informative Messages, Unique-Message Accuracy, Literal Consistency, Median Response Time, Choices Answered, Choices Set, Answer Rate and Anticipations. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Reference Game used in research?
81 papers from 2021 to 2025 name it in their title or abstract, up 45% compared with 2016 to 2020. That ranks it 273 of 582 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Reference Game have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Choices in each role (12 choices, 24 choices (standard)). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.
Related tests