ConductCognition
  • Domains

    Tests

    • Processing speed55
    • Attention143
    • Working memory51
    • Executive function191
    • Episodic memory84
    • Response inhibition10
    • Social cognition97
    • Language50

    Most used

    Evidence

    1. Verbal Fluency TestGradeA
    2. Stroop Color-Word TestGradeA
    3. Trail Making TestGradeA
    4. N-Back TaskGradeA
    5. Go/No-Go TaskGradeA
    6. Digit Span TestGradeA
    7. Symbol Coding TestGradeA
    8. Verbal Learning TestGradeA
    REDTry it nowStroop Color-Word Test
    All 681 tests

    Browse by

    • Phone380
    • Tablet415
    • Untimed419
    • Up to 5 minutes81
    • Evidence grade A48
  • First studies

    Grade, length

    • Reaction Time StudyGradeA7 min
    • Pilot ScreeningGradeA15 min
    • Medication StudiesGradeA−26 min
    • Language-Minimized ScreeningGradeA−30 min

    Age groups

    • Older Adult Executive SpeedGradeA20 min
    • School-AgeGradeA−33 min

    Memory

    • At-Home MemoryGradeA26 min
    • Working Memory and LearningGradeA−19 min
    • Associative MemoryGradeB32 min
    • Working Memory CapacityGradeB34 min

    Language and perception

    • LanguageGradeA−33 min
    • Visual PerceptionGradeB27 min
    • Listening and Auditory ProcessingGradeB31 min

    Executive function and attention

    • Executive ControlGradeA18 min
    • Attention and ImpulsivityGradeA−26 min
    • Visual AttentionGradeB27 min
    • Creativity and Flexible ThinkingGradeB30 min
    • Reasoning and SpatialGradeB32 min

    Movement

    • Motor and AttentionGradeA−29 min
    • Motor LearningGradeB34 min

    Behavior, mood and choice

    • Food Choice and Self-ControlGradeA−29 min
    • Sleep and FatigueGradeB23 min
    • Social CognitionGradeB23 min
    • Impulsivity and Substance UseGradeB26 min
    • Emotion and MoodGradeB28 min
    • Decisions and RewardGradeB29 min
    All 26 batteries

    Find one

    Up to 15 minutes2Up to 30 minutes19Phone8
    Test order labBattery gradesHow batteries work
  • Use and evidence

    • Overview
    • Compare all 681 tests
    • Test-retest
    • Reliability and validity
    • Battery grades26
    • Download and cite

    Clinical trials

    • Overview
    • By condition25
    • By test103
    • Trial design benchmark
    • Cognitive outcomes, 2026

    Conditions

    Registered studies

    • Dementia and Alzheimer's1,673
    • Aging, frailty and exercise1,400
    • Depression and bipolar disorder1,249
    • Stroke1,237
    • Substance use and addiction904
    • Parkinson's disease894
    • Cancer762
    All 25 conditions
    16,526Registered studies name a cognitive test in an outcome measure11,848 name a test you can run hereExplore the studies
  • 1

    Build

    • Study design
    • Customize tests
    • Repeat visits
    • Study cost calculator
    2

    Recruit

    • Recruitment setup
    • Prolific
    • Sona Systems
    • CloudResearch Connect
    • Qualtrics
    3

    Run

    • Remote testing
    • Participant devices
    • Phone timing check
    • Offline edition
    4

    Results

    • Automated scoring
    • Test norms
    • Study Quality Monitor
    • Data export
    Platform overview

    Compliance

    • HIPAA-ready testing
    • US clinical research
    • Data Policy
    • Privacy Policy
    • Terms of Use
  • Research areas

    Tests used in each

    • Aging and dementia202
    • Decisions and reward136
    • Child development and education114
    • Social behavior87
    • Language and bilingualism80
    • Psychiatric groups35
    • Training and intervention31
    • Mood, anxiety and trauma28
    • Substance use and impulsivity26
    • Neurological and movement research21
    • Sleep and fatigue17
    • Brain injury and concussion14
    • ADHD11
    • Eating and obesity11
    • Medication and drug trials8

    Research settings

    • Student participant pools
    • Teaching a class
    • Clinical trials
    • US clinical research
    • Sites without internet
    Planning a study?
  • Methods

    • How evidence grades work
    • How test batteries work
    • How test order works
    Methods and questions

    Help

    • FAQ
    • Recruitment guides
    • Compare platforms

    Updates

    • Blog
    • New tests by email
    Latest post10 November 2026What you can do in ConductCognition todayA short tour of ConductCognition as it stands on 10 November. After this, we'll post when something changes how you run a study.Read the post
  • Pricing
Start Free
Log In
Log InStart Free
  1. Home
  2. /Tests
  3. /Arithmetic Stories Task

Reasoning

Arithmetic Stories Task

Also called: arithmetic word problems, math word problems, compare problems, consistent and inconsistent language problems, consistency effect task

Measure arithmetic word problem solving: participants type the answers to short one- and two-step stories whose key word matches or contradicts the operation, scored as accuracy by wording and the share of wrong answers that follow the key word.

6-10 minUntimed

Create a study with this test

Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.

Papers, 2021 to 2025
520
Evidence grade
C
Registered clinical studies
1
How these are counted

The red jar holds 34 buttons. It holds 8 more buttons than the blue jar. How many buttons does the blue jar hold?

The red jar holds 34 buttons. It holds 8 more buttons than the blue jar. How many buttons does the blue jar hold?

26
Accuracy by wording42 follows the key word: more
Read a short story and type the number that answers its question
  • Overview
  • Evidence
  • Task design
  • Quality checks
  • Settings
  • Cite
  • Questions

What it measures

Measures arithmetic word problem solving. Participants read short stories and type the number that answers each one's question, in one or two steps. Half the stories word their comparison so that its key word (more, less) matches the operation and half so that it contradicts it. The main score is accuracy; the share of wrong answers that follow the key word is scored too.

Constructs

  • Arithmetic word problem solving
  • Comprehension of relational statements
  • Problem representation
  • Key-word strategy

Research uses

  • Mathematics education research
  • Numerical cognition research
  • Reading comprehension research
  • Comparing consistent and inconsistent wordings within a study

How the paradigm works

Arithmetic word problems measure how people turn a short story into arithmetic. In compare problems a relational sentence states one amount in terms of another, and solvers make more reversal errors, adding where they should subtract or the reverse, when its key word (more than, less than) contradicts the operation the story needs than when it matches it (Lewis and Mayer, 1987). With university students their account held for two-step problems, which put more demand on the solver, and not for one-step ones (Verschaffel, De Corte and Pauwels, 1992), and unsuccessful solvers tend to build their answer from the numbers and key words rather than from a model of the situation (Hegarty, Mayer and Monk, 1995). In this task people read 16 short stories written in each language, 8 one-step and 8 two-step that multiply after the comparison, and type the number that answers each. Every story is worded so that its key word matches the operation in some sessions and contradicts it in others.

Scores

Accuracy%
Right answers over the stories shown, 16 in a complete run.
Higher is better
One-Step Accuracy%
Accuracy on the 8 one-step stories.
Higher is better
Two-Step Accuracy%
Accuracy on the 8 two-step stories, which multiply after the comparison.
Higher is better
Consistent Wording Accuracy%
Accuracy on the 8 stories whose key word matches the operation.
Higher is better
Inconsistent Wording Accuracy%
Accuracy on the 8 stories whose key word contradicts the operation.
Higher is better
Consistency Effect%
Consistent minus inconsistent accuracy.
Higher means the contradicting wording cost more
Accuracy by Steps and Wording%
Accuracy on the one-step and two-step stories in each wording, such as twoStep-inconsistent.
Lower on inconsistent stories means the wording mattered
Key-Word Answers%
Share of the wrong typed answers that follow the key word instead of the story: adding for more, or subtracting for less, where the story needs the opposite.
Higher means more answers built from the key word
Reversals, Consistent Wording%
Answers with the comparison reversed, over the consistent stories.
Usually low
Reversals, Inconsistent Wording%
Answers with the comparison reversed, over the inconsistent stories.
Higher than on consistent stories means the wording drew reversals
Key-Word Errorscount
Inconsistent stories answered with the operation their key word suggests.
Informational
Reversals Against the Key Wordcount
Consistent stories answered with the opposite of the operation their key word suggests.
Informational
First Step Onlycount
Two-step stories answered with the result of the comparison alone, in either direction.
Informational
Other Errorscount
Wrong typed numbers that match no named error, such as slips in the arithmetic.
Informational
Unanswered Storiescount
Stories left blank or without a number that can be read.
Quality check
Median Answer Timems
Median time from a story appearing to its answer, for answers of at least 100 ms.
Informational; the task has no time limit
Median Time, Consistent Wordingms
Median time to a right answer on the consistent stories, for answers of at least 100 ms.
Informational
Median Time, Inconsistent Wordingms
Median time to a right answer on the inconsistent stories, for answers of at least 100 ms.
Longer than on consistent stories means the wording took extra work
Answers Under 100 mscount
Answers submitted less than 100 ms after their story appeared.
0 expected
Stories Answeredcount
Stories with a typed number that can be read.
Fewer than half the planned stories invalidates
Storiescount
Stories shown in the scored run.
Compare with planned stories
Planned Storiescount
16: 8 one-step and 8 two-step, half of each in each wording.
Informational
Most Repeated Answercount
The largest number of stories given one identical number.
Half the planned stories or more is flagged

Use and evidence

520 papers from 2021 to 2025 name this task in their title or abstract, up 14% compared with 2016 to 2020. That ranks it 56 of 711 test families in our census. See the counts for every test.

2021 to 2025, the years gradedEarlier years
Papers per year naming the Arithmetic Stories Task, 2000 to 2025Bar chart. 2021: 71; 2022: 98; 2023: 89; 2024: 132; 2025: 130 papers. Peak year 2024.0501001502000: 14 papers2001: 16 papers2002: 19 papers2003: 30 papers2004: 26 papers2005: 34 papers2006: 24 papers2007: 35 papers2008: 39 papers2009: 51 papers2010: 61 papers2011: 56 papers2012: 61 papers2013: 79 papers2014: 72 papers2015: 65 papers2016: 75 papers2017: 76 papers2018: 75 papers2019: 127 papers2020: 105 papers2021: 71 papers2022: 98 papers2023: 89 papers2024: 132 papers2025: 130 papers200020052010201520202025
Papers per year naming the Arithmetic Stories Task, 2000 to 2025Bar chart. 2021: 71; 2022: 98; 2023: 89; 2024: 132; 2025: 130 papers. Peak year 2024.0501001502000: 14 papers2001: 16 papers2002: 19 papers2003: 30 papers2004: 26 papers2005: 34 papers2006: 24 papers2007: 35 papers2008: 39 papers2009: 51 papers2010: 61 papers2011: 56 papers2012: 61 papers2013: 79 papers2014: 72 papers2015: 65 papers2016: 75 papers2017: 76 papers2018: 75 papers2019: 127 papers2020: 105 papers2021: 71 papers2022: 98 papers2023: 89 papers2024: 132 papers2025: 130 papers200020052010201520202025
Papers per year whose title or abstract names this task (OpenAlex).
Show the numbers
YearPapers
2025130
2024132
202389
202298
202171
2020105
2019127
201875
201776
201675
201565
201472
201379
201261
201156
201061
200951
200839
200735
200624
200534
200426
200330
200219
200116
200014
B

Use grade B

300 to 999 papers from 2021 to 2025.

C

Evidence grade C

At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.

Papers on test-retest reliability
1
Papers on reliability or validity
110
Registered clinical studies using it as an outcome
1

Measured on ConductCognition

Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.

No pooled figure yet.

Note on the counts: mostly classroom and math education studies.

How the grades work

Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.

Source: ConductCognition test census 0.58.0, counted 14 November 2026 from OpenAlex and ClinicalTrials.gov.

Task design

The trial sequence as the test runs in ConductCognition.

Stories
16 stories written for this library in each language: 8 one-step and 8 two-step, which multiply after the comparison, such as how many points 4 purple tickets cost if an orange ticket costs 12 points and a purple one 3 points more.
Wording
Each story relates one amount to another by more or less. In the consistent wording the key word matches the operation; in the inconsistent wording it contradicts it, as when the orange ticket costs 3 points more than a purple one. The seed gives half the stories of each kind each wording, so every story appears in both across sessions.
Answers
Typed numbers, all whole and under 100 before the second step. One practice story with feedback, then no time limit and no feedback.
Order
The 16 stories in an order drawn from the session's seed.
Scores
Accuracy (the main score), by steps and by wording, the consistency effect, and the share of wrong answers that follow the key word instead of the story. Reversals and answers that stop after the first step are counted.

The stories and numbers are this library's own. The scores describe answers to these stories, support research and make no grade-level, school-readiness or diagnostic claim.

Quality checks

Each session is marked valid, needs review or invalid, with the reason, before you see its scores.

Scored rows: word_problems_story from the run with the most of them, latest on ties, one typed number per story. Typed answers have no chance level, so accuracy is not judged against chance, and stories have no time limit, so none can be missed. Fewer than half the planned stories answered invalidates. An incomplete run is flagged, and so is one number typed for half the stories or more, and right answers on fewer than half the stories whose key word matches the operation. Answers under 200 ms on at least half of four or more answered stories invalidate. A hidden tab is flagged.

Why these rules

There is no correct option to compare with chance, so accuracy is not judged against chance. A score over fewer than half the stories cannot be compared with a score over all of them, so it invalidates. Wrong answers that follow the key word are what the test measures and never invalidate. The stories have different answers, so one number typed for half of them cannot follow them; it is flagged. On the consistent stories the key word and the story agree, so missing most of them means the stories or the arithmetic were not followed; it is flagged. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. Stories stay on screen until they are answered, so a hidden tab does not change what was shown, but it can mean a calculator or help was used; it is flagged.

Running this test

Settings you can change

This test runs with fixed settings.

How test settings work

Need a different version of this test?

Devices and languages

  • Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
  • Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
  • Available on: Every plan, including the free plan.
  • Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.

Cite and describe

Methods text, with the standard settings

Participants completed the Arithmetic Stories Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Lewis & Mayer, 1987; Verschaffel, De Corte & Pauwels, 1992; Hegarty, Mayer & Monk, 1995). The main score was accuracy (right answers over the stories shown, 16 in a complete run).

ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.

  • Lewis, A. B., & Mayer, R. E. (1987). Students' miscomprehension of relational statements in arithmetic word problems. Journal of Educational Psychology, 79(4), 363-371. https://doi.org/10.1037/0022-0663.79.4.363Compare problems in consistent and inconsistent language, a compare step and then a second step, where college students made more reversal errors in the inconsistent form.
  • Verschaffel, L., De Corte, E., & Pauwels, A. (1992). Solving compare problems: An eye movement test of Lewis and Mayer's consistency hypothesis. Journal of Educational Psychology, 84(1), 85-94. https://doi.org/10.1037/0022-0663.84.1.85One-step and two-step compare problems with university students and third graders, where the consistency account held when the task put demands on the solver.
  • Hegarty, M., Mayer, R. E., & Monk, C. A. (1995). Comprehension of arithmetic word problems: A comparison of successful and unsuccessful problem solvers. Journal of Educational Psychology, 87(1), 18-32. https://doi.org/10.1037/0022-0663.87.1.18Unsuccessful solvers building their plan from numbers and key words, successful ones from a model of the situation.

Source papers

    Questions

    How long does the Arithmetic Stories Task take?

    About 6 to 10 minutes, as listed in the test library.

    What device does a participant need?

    Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.

    What scores does it produce?

    Accuracy, One-Step Accuracy, Two-Step Accuracy, Consistent Wording Accuracy, Inconsistent Wording Accuracy, Consistency Effect, Accuracy by Steps and Wording, Key-Word Answers, Reversals, Consistent Wording, Reversals, Inconsistent Wording, Key-Word Errors, Reversals Against the Key Word, First Step Only, Other Errors, Unanswered Stories, Median Answer Time, Median Time, Consistent Wording, Median Time, Inconsistent Wording, Answers Under 100 ms, Stories Answered, Stories, Planned Stories and Most Repeated Answer. Each session is also marked valid, needs review or invalid, with the reason.

    How widely is the Arithmetic Stories Task used in research?

    520 papers from 2021 to 2025 name it in their title or abstract, up 14% compared with 2016 to 2020. That ranks it 56 of 711 test families in the ConductCognition test census (use grade B). Source: ConductCognition test census 0.58.0, counted 14 November 2026 from OpenAlex and ClinicalTrials.gov.

    How much reliability evidence does the Arithmetic Stories Task have?

    In the published research, 1 paper reports its test-retest reliability and 1 registered clinical study uses it as an outcome (evidence grade C in the ConductCognition test census).

    Which languages can participants use?

    Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.

    Is this a clinical test?

    No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.

    Every test: the full test list.

    New tests and features, about once a month. Unsubscribe at any time.

    Run this test in your next study

    Start free with 10 free participants on any test, then use participant credits or a yearly plan.

    Start free

    Related tests

    Proportional Reasoning TaskNumeracy Problems TestArithmetic Fluency Task
    ConductCognition

    Hosted browser-native cognitive testing for academic and commercial research workflows.

    A product of ConductScience, Inc.

    Tests

    • All cognitive tests
    • Test batteries
    • Use and evidence
    • Test norms
    • Trail Making Test
    • N-Back Task
    • Big Five personality test
    • Automated scoring
    • Study design
    • New tests by email

    Guides

    • Methods and questions
    • Blog
    • Recruitment setup
    • Participant devices
    • Prolific
    • Sona Systems
    • CloudResearch Connect
    • Qualtrics

    Use cases

    • Student participant pools
    • Aging and dementia research
    • Clinical trials
    • US clinical research
    • Remote testing

    Compare and buy

    • Pricing
    • Compare platforms
    • Offline edition
    • FAQ

    Trust

    • Study Quality Monitor
    • HIPAA-ready testing
    • Data export
    • Privacy Policy
    • Terms of Use
    • Data Policy

    Company

    • ConductScience
    • ConductScience Fellows
    © 2026 ConductScience, Inc. Research use only.