Reasoning
Numeracy Problems Test
Also called: objective numeracy test, statistical numeracy problems, risk literacy problems
Measure numeracy: participants type the answers to short problems on percentages, chances and risks, written for this test in eight languages.
8-12 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 87
- Evidence grade
- C
- Registered clinical studies
- 13
What it measures
Measures numeracy: working with percentages, chances and risks. Participants type the answers to 12 short problems written for this test, from turning a percentage into a count to judging how much a change lowers a risk. The main score is the number answered correctly. The problems are written in each of the eight languages.
Constructs
- Numeracy
- Probability and risk reasoning
- Applied arithmetic
Research uses
- Risk communication and decision research
- Individual differences in numeracy
- Within-study group comparison
How the paradigm works
Numeracy is the ability to understand and use numbers, chances and risks. Objective numeracy tests give people short problems to work out, such as how many of 1,000 people a 10% risk means, and count the correct answers (Schwartz, Woloshin, Black and Welch 1997; Lipkus, Samsa and Rimer 2001). Lower numeracy goes with poorer understanding of risk information (Reyna and colleagues 2009), more numerate people are less swayed by how numbers are framed (Peters and colleagues 2006), and harder problems on conditional chances spread out highly educated samples (Cokely and colleagues 2012).
More about the paradigm
The 12 problems here were written for this test: 4 on percentages and counts, 4 on chances and 4 on reading risks, from turning a percentage into a count to the chance behind a positive signal given as counts (Gigerenzer and Hoffrage 1995). Every answer is a whole number from 1 to 99. The problems are written in each of the eight languages, with their numbers in the language's format. They come in three bands of rough difficulty, easier to harder, and the session's random seed orders the problems within each band; the order is recorded. One practice problem with feedback shows the answer box. Participants type each answer, without a calculator, with no time limit and no feedback. A % sign sits beside the box when the answer is a percent, and words around a number are ignored, so "about 30" counts as 30.
The main score is the number of problems answered correctly, as in the published scales. Scores for each kind of problem, wrong and blank answers and the time per problem are also reported. Scores support research and do not diagnose.
Scores
- Problems Correctitems
- Problems answered with the correct number, of 12.
- Higher means more correct answers
- Percentage Problems Correctitems
- Correct answers among the 4 problems on percentages and counts.
- Higher means more correct answers
- Chance Problems Correctitems
- Correct answers among the 4 problems on chances.
- Higher means more correct answers
- Risk Problems Correctitems
- Correct answers among the 4 problems on reading risks.
- Higher means more correct answers
- Wrong Answersitems
- Answers that were not the correct number, including answers with no number in them.
- Higher means more errors
- Blank Answersitems
- Problems submitted with an empty box.
- Lower means more problems were attempted
- Median Time per Problemms
- Median time from a problem's appearance to its submission, over answered problems, leaving out answers under 100 ms.
- Describes pace; the test has no time limit
- Answers Under 100 mscount
- Answered problems submitted within 100 ms.
- Expected to be 0
- Problems Answereditems
- Problems submitted with an answer.
- A session with fewer than 6 is invalid
- Problems Shownitems
- Problems presented in the scored run.
- 12 in a completed run
- Largest Same Answer Countitems
- The largest number of problems given one identical answer.
- 6 or more is flagged for review
Use and evidence
87 papers from 2021 to 2025 name this task in their title or abstract, about the same as from 2016 to 2020. That ranks it 199 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 21 |
| 2024 | 12 |
| 2023 | 13 |
| 2022 | 19 |
| 2021 | 22 |
| 2020 | 28 |
| 2019 | 14 |
| 2018 | 20 |
| 2017 | 14 |
| 2016 | 11 |
| 2015 | 19 |
| 2014 | 16 |
| 2013 | 13 |
| 2012 | 16 |
| 2011 | 6 |
| 2010 | 7 |
| 2009 | 2 |
| 2008 | 1 |
| 2007 | 4 |
| 2006 | 0 |
| 2005 | 0 |
| 2004 | 0 |
| 2003 | 0 |
| 2002 | 0 |
| 2001 | 1 |
| 2000 | 0 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 6
- Papers on reliability or validity
- 46
- Registered clinical studies using it as an outcome
- 13
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Problems
- 12 problems written for this test: 4 on percentages and counts, 4 on chances and 4 on reading risks. Every answer is a whole number from 1 to 99, and no two answers are the same.
- Order
- 3 bands of rough difficulty, 4 problems each, run from easier to harder. The session's seed orders the problems within each band, and the order is saved.
- Languages
- Each problem is written in all eight languages, with its numbers passed in so a translation cannot drop one, and numbers follow the language's format (1,000 or 1.000).
- Answer
- A typed number, in a box that opens the number keypad on phones, with a % sign beside it when the answer is a percent. Words around the number are ignored, and 1 in 25 reads as 25. No calculator, no time limit and no feedback.
- Practice
- 1 problem outside the set, with feedback.
The problems were written for this test; no published numeracy item is used or reworded, so published norms do not apply.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: numeracy_problems_problem from the scored run, one typed number per problem. Typed answers have no chance level, so accuracy is not judged against chance, and problems have no time limit, so none can be missed. Fewer than 6 of the 12 problems answered invalidates. Typed answers under 200 ms on at least half of 4 or more answered problems invalidate. One identical answer on 6 or more problems is flagged for review. A hidden tab is flagged.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. A score over fewer than half the problems cannot be compared with a score over all of them, so it invalidates. A low score is the result the test measures and never invalidates. The 12 answers all differ, so one answer typed for half the problems or more cannot follow them; it is flagged. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. Problems stay on screen until they are submitted, so a hidden tab does not change what was shown, but it can mean a calculator or help was used; it is flagged.
Running this test
Settings you can change
This test runs with fixed settings.
Need a different version of this test?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Numeracy Problems Test in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Schwartz et al., 1997; Lipkus, Samsa & Rimer, 2001; Cokely et al., 2012). The main score was problems correct (problems answered with the correct number, of 12).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Schwartz, L. M., Woloshin, S., Black, W. C., & Welch, H. G. (1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127(11), 966-972. https://doi.org/10.7326/0003-4819-127-11-199712010-00003
- Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21(1), 37-44. https://doi.org/10.1177/0272989X0102100105
- Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17(5), 407-413. https://doi.org/10.1111/j.1467-9280.2006.01720.x
- Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring risk literacy: The Berlin Numeracy Test. Judgment and Decision Making, 7(1), 25-47. https://doi.org/10.1017/S1930297500001819
- Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684-704. https://doi.org/10.1037/0033-295X.102.4.684
Source papers
- Lipkus IM, Samsa G, Rimer BK (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21(1):37-44.Paradigm source, Adults
- Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia-Retamero R (2012). Measuring risk literacy: The Berlin Numeracy Test. Judgment and Decision Making, 7(1):25-47.Paradigm source, Adults
Questions
How long does the Numeracy Problems Test take?
About 8 to 12 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Problems Correct, Percentage Problems Correct, Chance Problems Correct, Risk Problems Correct, Wrong Answers, Blank Answers, Median Time per Problem, Answers Under 100 ms, Problems Answered, Problems Shown and Largest Same Answer Count. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Numeracy Problems Test used in research?
87 papers from 2021 to 2025 name it in their title or abstract, about the same as from 2016 to 2020. That ranks it 199 of 395 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Numeracy Problems Test have?
In the published research, 6 papers report its test-retest reliability and 13 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.