ConductCognition
  • Domains

    Tests

    • Processing speed55
    • Attention143
    • Working memory51
    • Executive function191
    • Episodic memory84
    • Response inhibition10
    • Social cognition97
    • Language50

    Most used

    Evidence

    1. Verbal Fluency TestGradeA
    2. Stroop Color-Word TestGradeA
    3. Trail Making TestGradeA
    4. N-Back TaskGradeA
    5. Go/No-Go TaskGradeA
    6. Digit Span TestGradeA
    7. Symbol Coding TestGradeA
    8. Verbal Learning TestGradeA
    REDTry it nowStroop Color-Word Test
    All 681 tests

    Browse by

    • Phone380
    • Tablet415
    • Untimed419
    • Up to 5 minutes81
    • Evidence grade A48
  • First studies

    Grade, length

    • Reaction Time StudyGradeA7 min
    • Pilot ScreeningGradeA15 min
    • Medication StudiesGradeA−26 min
    • Language-Minimized ScreeningGradeA−30 min

    Age groups

    • Older Adult Executive SpeedGradeA20 min
    • School-AgeGradeA−33 min

    Memory

    • At-Home MemoryGradeA26 min
    • Working Memory and LearningGradeA−19 min
    • Associative MemoryGradeB32 min
    • Working Memory CapacityGradeB34 min

    Language and perception

    • LanguageGradeA−33 min
    • Visual PerceptionGradeB27 min
    • Listening and Auditory ProcessingGradeB31 min

    Executive function and attention

    • Executive ControlGradeA18 min
    • Attention and ImpulsivityGradeA−26 min
    • Visual AttentionGradeB27 min
    • Creativity and Flexible ThinkingGradeB30 min
    • Reasoning and SpatialGradeB32 min

    Movement

    • Motor and AttentionGradeA−29 min
    • Motor LearningGradeB34 min

    Behavior, mood and choice

    • Food Choice and Self-ControlGradeA−29 min
    • Sleep and FatigueGradeB23 min
    • Social CognitionGradeB23 min
    • Impulsivity and Substance UseGradeB26 min
    • Emotion and MoodGradeB28 min
    • Decisions and RewardGradeB29 min
    All 26 batteries

    Find one

    Up to 15 minutes2Up to 30 minutes19Phone8
    Test order labBattery gradesHow batteries work
  • Use and evidence

    • Overview
    • Compare all 681 tests
    • Test-retest
    • Reliability and validity
    • Battery grades26
    • Download and cite

    Clinical trials

    • Overview
    • By condition25
    • By test103
    • Trial design benchmark
    • Cognitive outcomes, 2026

    Conditions

    Registered studies

    • Dementia and Alzheimer's1,673
    • Aging, frailty and exercise1,400
    • Depression and bipolar disorder1,249
    • Stroke1,237
    • Substance use and addiction904
    • Parkinson's disease894
    • Cancer762
    All 25 conditions
    16,526Registered studies name a cognitive test in an outcome measure11,848 name a test you can run hereExplore the studies
  • 1

    Build

    • Study design
    • Customize tests
    • Repeat visits
    • Study cost calculator
    2

    Recruit

    • Recruitment setup
    • Prolific
    • Sona Systems
    • CloudResearch Connect
    • Qualtrics
    3

    Run

    • Remote testing
    • Participant devices
    • Phone timing check
    • Offline edition
    4

    Results

    • Automated scoring
    • Test norms
    • Study Quality Monitor
    • Data export
    Platform overview

    Compliance

    • HIPAA-ready testing
    • US clinical research
    • Data Policy
    • Privacy Policy
    • Terms of Use
  • Research areas

    Tests used in each

    • Aging and dementia202
    • Decisions and reward136
    • Child development and education114
    • Social behavior87
    • Language and bilingualism80
    • Psychiatric groups35
    • Training and intervention31
    • Mood, anxiety and trauma28
    • Substance use and impulsivity26
    • Neurological and movement research21
    • Sleep and fatigue17
    • Brain injury and concussion14
    • ADHD11
    • Eating and obesity11
    • Medication and drug trials8

    Research settings

    • Student participant pools
    • Teaching a class
    • Clinical trials
    • US clinical research
    • Sites without internet
    Planning a study?
  • Methods

    • How evidence grades work
    • How test batteries work
    • How test order works
    Methods and questions

    Help

    • FAQ
    • Recruitment guides
    • Compare platforms

    Updates

    • Blog
    • New tests by email
    Latest post10 November 2026What you can do in ConductCognition todayA short tour of ConductCognition as it stands on 10 November. After this, we'll post when something changes how you run a study.Read the post
  • Pricing
Start Free
Log In
Log InStart Free
  1. Home
  2. /Tests
  3. /Probability Words Task

Executive Function

Probability Words Task

Also called: verbal probability expressions, words of estimative probability, probability phrases, verbal probabilities, numerical translation of probability words

Measure what probability words mean in numbers: participants give the chance that 12 words such as likely or rare mean to them, alone and in sentences, and type each word's range, scored against the IPCC likelihood scale.

6-10 minUntimed

Create a study with this test

Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.

Papers, 2021 to 2025
64
Evidence grade
D
Registered clinical studies
1
How these are counted
likely
70%
Lowest and highest chance for “likely”
55%85%
Inside the IPCC rangeShift toward 50%
Give the chance each probability word means to you, alone and in a sentence, then type its range
  • Overview
  • Evidence
  • Task design
  • Quality checks
  • Settings
  • Cite
  • Questions

What it measures

Measures what probability words such as likely, rare and almost certain mean in numbers. Participants give the chance each of 12 words means to them on a slider from 0 to 100, alone and in a short sentence written for this test in each of the eight languages, then type the range each word covers. The main score is the share of answers to the 7 terms of the IPCC's likelihood scale that fall in the IPCC's range.

Constructs

  • Verbal probability
  • Risk communication
  • Uncertainty communication
  • Numeracy
  • Vagueness

Research uses

  • Judgment and decision-making research
  • Risk and uncertainty communication research
  • Forecasting and climate communication research
  • Cross-language comparison of probability words

How the paradigm works

The Probability Words Task measures what verbal probability expressions, such as likely, rare or almost certain, mean in numbers to each person, a question for risk labels, forecasts and climate reports, since people read the same words very differently (Wallsten and colleagues, 1986; Budescu, Broomell and Por, 2009). Twelve words and phrases, written for this task in each language, span the scale from almost impossible to virtually certain; seven are terms of the likelihood scale in the IPCC's guidance for its Fifth Assessment Report, which gives each a range (likely, 66 to 100%; very unlikely, 0 to 10%). Each word comes alone and in one short sentence about an everyday event, in one order drawn from the session's seed, and the participant moves a slider from 0 to 100 to the chance it means to them. Then each word comes again and the participant types the lowest and the highest chance it can mean. A setting shows the IPCC range beside each term in the sentences, the format Budescu and colleagues proposed. There is no time limit and no feedback.

More about the paradigm

The main score is the share of answers to the IPCC terms, alone and in sentences, that fall inside the IPCC range. Beside it come the mean distance outside the range in percentage points and how far answers move toward 50% from each term's midpoint (laypeople read the IPCC's terms as closer to 50% than intended; Budescu and colleagues, 2014). Also reported are the share of pairs of terms answered in the scale's order, the mean difference between a word alone and in its sentence, the median width of the typed ranges and the share of answers inside the person's own range. Every answer names its word, so a study can measure the spread across its participants for each word.

Scores

Consistency With the IPCC Scale%
Share of answers to the IPCC terms, alone and in sentences, inside the IPCC range, ends included.
Higher means closer to the IPCC's ranges
Distance From the IPCC Scale%
Mean distance of those answers outside the IPCC range, in percentage points; 0 inside.
Lower is closer
Shift Toward 50%%
Mean of how much closer to 50% an answer is than its term's IPCC midpoint, for terms whose range leaves out 50%.
Above 0, answers ran toward the middle of the scale
Order Agreement%
Share of pairs of IPCC terms, answered alone, in the scale's order.
1 keeps the order; 0.2 or less is flagged
Sentence Shift%
Mean absolute difference between a word's answer alone and in its sentence, in percentage points.
Informational
Median Range Width%
Median of the highest minus the lowest typed chance.
Wider means the words cover more chances
Answers Inside Own Range%
Share of words whose answer alone falls inside the person's own range for it.
Informational
Median Answer Timems
Median time from a screen appearing to its answer, for answers of at least 100 ms.
Informational; the task has no time limit
Answers Under 100 mscount
Answers given less than 100 ms after their screen appeared.
0 expected
Words Answeredcount
Words answered alone in the scored run.
Fewer than half the words and sentences together invalidates
Sentences Answeredcount
Sentences answered in the scored run.
Informational
Ranges Answeredcount
Words with a typed range in the scored run.
Informational
Planned Wordscount
Words the run planned: 12, each alone, in a sentence and with a range.
Informational

Use and evidence

64 papers from 2021 to 2025 name this task in their title or abstract, up 21% compared with 2016 to 2020. That ranks it 402 of 711 test families in our census. See the counts for every test.

2021 to 2025, the years gradedEarlier years
Papers per year naming the Probability Words Task, 2000 to 2025Bar chart. 2021: 12; 2022: 12; 2023: 9; 2024: 10; 2025: 21 papers. Peak year 2025.01020302000: 6 papers2001: 3 papers2002: 7 papers2003: 2 papers2004: 7 papers2005: 2 papers2006: 9 papers2007: 1 papers2008: 7 papers2009: 10 papers2010: 7 papers2011: 11 papers2012: 5 papers2013: 14 papers2014: 8 papers2015: 11 papers2016: 8 papers2017: 9 papers2018: 5 papers2019: 18 papers2020: 13 papers2021: 12 papers2022: 12 papers2023: 9 papers2024: 10 papers2025: 21 papers200020052010201520202025
Papers per year naming the Probability Words Task, 2000 to 2025Bar chart. 2021: 12; 2022: 12; 2023: 9; 2024: 10; 2025: 21 papers. Peak year 2025.01020302000: 6 papers2001: 3 papers2002: 7 papers2003: 2 papers2004: 7 papers2005: 2 papers2006: 9 papers2007: 1 papers2008: 7 papers2009: 10 papers2010: 7 papers2011: 11 papers2012: 5 papers2013: 14 papers2014: 8 papers2015: 11 papers2016: 8 papers2017: 9 papers2018: 5 papers2019: 18 papers2020: 13 papers2021: 12 papers2022: 12 papers2023: 9 papers2024: 10 papers2025: 21 papers200020052010201520202025
Papers per year whose title or abstract names this task (OpenAlex).
Show the numbers
YearPapers
202521
202410
20239
202212
202112
202013
201918
20185
20179
20168
201511
20148
201314
20125
201111
20107
200910
20087
20071
20069
20052
20047
20032
20027
20013
20006
D

Use grade D

30 to 99 papers from 2021 to 2025.

D

Evidence grade D

Little published measurement evidence found.

Papers on test-retest reliability
1
Papers on reliability or validity
11
Registered clinical studies using it as an outcome
1

Measured on ConductCognition

Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.

No pooled figure yet.

How the grades work

Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.

Source: ConductCognition test census 0.58.0, counted 13 November 2026 from OpenAlex and ClinicalTrials.gov.

Task design

The trial sequence as the test runs in ConductCognition.

Words
12 probability words and phrases written for this library in each language, from almost impossible to virtually certain. 7 are terms of the IPCC likelihood scale, which gives each a range, such as likely, 66 to 100%; the other 5, such as rare and doubtful, are everyday words it does not define.
Answer
Each word alone and in one short sentence about an everyday event, such as a train arriving on time: a slider from 0 to 100% that starts with no value chosen. Then each word again, with the lowest and the highest chance it can mean typed in two boxes.
Order
The 24 slider answers in one order drawn from the session's seed, then the 12 ranges in another. One practice word comes first; there is no feedback and no time limit.
Setting
The IPCC ranges in the sentences: hidden (standard) or shown beside each IPCC term, the format Budescu and colleagues proposed. The words alone and the ranges never show them.
Scores
The share of answers to the IPCC terms inside the IPCC range (the main score); their mean distance outside it; the shift toward 50%; the share of pairs of terms in the scale's order; the mean difference between a word alone and in its sentence; the median range width; and the share of answers inside the person's own range.

Every answer names its word, so a study can measure how much its participants differ on each word. The words and sentences are this library's own in each language, and the IPCC defined its ranges for its English terms. The scores describe how words are read; they support research and do not judge a person.

Quality checks

Each session is marked valid, needs review or invalid, with the reason, before you see its scores.

Scored rows: probability_words_item from the run with the most of them, latest on ties, and that run's probability_words_range rows. A chance for a word has no right answer, and answers far from the IPCC's ranges are part of what the task measures, so they never invalidate. Fewer than half the planned words and sentences answered invalidates, and any other incomplete run is flagged. IPCC terms answered alone against the scale's order on at least 80% of their pairs are flagged for review. Answers under 100 ms are counted as anticipations, and answers under 200 ms on at least half the answers invalidate. A hidden tab withholds the percentile.

Why these rules

There is no correct option to compare with chance, so accuracy is not judged against chance. People read the same probability words very differently, so an answer outside a term's IPCC range is a result, not a failed task. Answers that put "very unlikely" above "very likely" on most pairs suggest the slider was read the wrong way round, so they are flagged and never exclude. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. A hidden tab could allow a search for published ranges, so the percentile is withheld after one.

No percentile. Scores and quality flags only.

Running this test

Settings you can change

SettingStandardAllowed
IPCC ranges in sentencesHidden (standard)Hidden (standard), Shown

Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.

How test settings work

Need a setting this test does not have?

Devices and languages

  • Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
  • Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
  • Available on: Every plan, including the free plan.
  • No percentile. Scores and quality flags only. How norms work
  • Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.

Cite and describe

Methods text, with the standard settings

Participants completed the Probability Words Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Wallsten TS, Budescu DV, Rapoport A, Zwick R, Forsyth B, 1986; Budescu DV, Broomell S, Por H-H, 2009), with the standard settings (ipcc ranges in sentences: Hidden). The main score was consistency with the IPCC scale (share of answers to the IPCC terms, alone and in sentences, inside the IPCC range, ends included).

ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.

  • Wallsten, T. S., Budescu, D. V., Rapoport, A., Zwick, R., & Forsyth, B. (1986). Measuring the vague meanings of probability terms. Journal of Experimental Psychology: General, 115(4), 348-365. https://doi.org/10.1037/0096-3445.115.4.348The vague meanings of probability terms such as doubtful, probable and likely as functions over the probability scale, with large but stable differences between people.
  • Budescu, D. V., Broomell, S., & Por, H.-H. (2009). Improving communication of uncertainty in the reports of the Intergovernmental Panel on Climate Change. Psychological Science, 20(3), 299-308. https://doi.org/10.1111/j.1467-9280.2009.02284.xReaders gave numbers to the probability terms in sentences from the 2007 IPCC report, and their numbers deviated from the IPCC's guidelines even with the guidelines at hand; the authors proposed another way to communicate uncertainty.
  • Budescu, D. V., Por, H.-H., Broomell, S. B., & Smithson, M. (2014). The interpretation of IPCC probabilistic statements around the world. Nature Climate Change, 4(6), 508-512. https://doi.org/10.1038/nclimate2194In 25 samples in 24 countries and 17 languages, laypeople read IPCC statements as closer to 50% than intended, and numerical ranges beside the terms brought readings closer to the guidelines.
  • Mastrandrea, M. D., Field, C. B., Stocker, T. F., Edenhofer, O., Ebi, K. L., Frame, D. J., Held, H., Kriegler, E., Mach, K. J., Matschoss, P. R., Plattner, G.-K., Yohe, G. W., & Zwiers, F. W. (2010). Guidance note for lead authors of the IPCC Fifth Assessment Report on consistent treatment of uncertainties. Intergovernmental Panel on Climate Change. The likelihood scale: virtually certain 99 to 100%, very likely 90 to 100%, likely 66 to 100%, about as likely as not 33 to 66%, unlikely 0 to 33%, very unlikely 0 to 10% and exceptionally unlikely 0 to 1%.

Source papers

  • Budescu DV, Broomell S, Por H-H (2009). Improving communication of uncertainty in the reports of the Intergovernmental Panel on Climate Change. Psychological Science, 20(3):299-308.Paradigm source
  • No normative data for this task.Paradigm source, N/A

Questions

How long does the Probability Words Task take?

About 6 to 10 minutes, as listed in the test library.

What device does a participant need?

Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.

What scores does it produce?

Consistency With the IPCC Scale, Distance From the IPCC Scale, Shift Toward 50%, Order Agreement, Sentence Shift, Median Range Width, Answers Inside Own Range, Median Answer Time, Answers Under 100 ms, Words Answered, Sentences Answered, Ranges Answered and Planned Words. Each session is also marked valid, needs review or invalid, with the reason.

How widely is the Probability Words Task used in research?

64 papers from 2021 to 2025 name it in their title or abstract, up 21% compared with 2016 to 2020. That ranks it 402 of 711 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.58.0, counted 13 November 2026 from OpenAlex and ClinicalTrials.gov.

How much reliability evidence does the Probability Words Task have?

In the published research, 1 paper reports its test-retest reliability and 1 registered clinical study uses it as an outcome (evidence grade D in the ConductCognition test census).

Does it report a percentile?

No. Scores and quality flags only.

Which settings can a researcher change?

Ipcc ranges in sentences (Hidden (standard), Shown). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.

Which languages can participants use?

Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.

Is this a clinical test?

No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.

Every test: the full test list.

New tests and features, about once a month. Unsubscribe at any time.

Run this test in your next study

Start free with 10 free participants on any test, then use participant credits or a yearly plan.

Start free

Related tests

Icon Array TaskNumeracy Problems TestConfidence Interval Task
ConductCognition

Hosted browser-native cognitive testing for academic and commercial research workflows.

A product of ConductScience, Inc.

Tests

  • All cognitive tests
  • Test batteries
  • Use and evidence
  • Test norms
  • Trail Making Test
  • N-Back Task
  • Big Five personality test
  • Automated scoring
  • Study design
  • New tests by email

Guides

  • Methods and questions
  • Blog
  • Recruitment setup
  • Participant devices
  • Prolific
  • Sona Systems
  • CloudResearch Connect
  • Qualtrics

Use cases

  • Student participant pools
  • Aging and dementia research
  • Clinical trials
  • US clinical research
  • Remote testing

Compare and buy

  • Pricing
  • Compare platforms
  • Offline edition
  • FAQ

Trust

  • Study Quality Monitor
  • HIPAA-ready testing
  • Data export
  • Privacy Policy
  • Terms of Use
  • Data Policy

Company

  • ConductScience
  • ConductScience Fellows
© 2026 ConductScience, Inc. Research use only.