Reasoning
Worked Examples Task
Also called: worked example effect task, worked-example effect paradigm, example-problem pairs, learning from worked examples, Sweller and Cooper paradigm
Measure learning from worked examples: participants learn chance problems by studying solved examples or by solving the same problems, then solve similar and new problems.
15-25 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 114
- Evidence grade
- C
- Registered clinical studies
- 0
What it measures
Measures learning from worked examples. Participants learn to solve chance problems, such as drawing marbles from a bag, either by studying worked examples, each followed by a similar practice problem, or by solving the same problems themselves, then solve 6 test problems: 3 like the practice problems and 3 new ones. The main score is the percent of test problems answered right. Mental effort is rated after each problem.
Constructs
- Learning from examples
- Cognitive load
- Problem-solving transfer
- Probability reasoning
Research uses
- Instructional design and learning research
- Cognitive load research
- Mathematics and statistics education research
- Within-study group comparison
How the paradigm works
The worked examples task measures learning from worked examples, a central finding of cognitive load research: people new to a kind of problem who study problems solved step by step, each followed by a similar problem to solve, often learn more than people who solve the same problems themselves. Sweller and Cooper (1985) showed it in algebra, and the review of Atkinson, Derry, Renkl and Wortham (2000) drew design rules from the research: pair each worked example with a practice problem of the same kind, give at least two problems of each kind, and lay the solution out in clear steps.
More about the paradigm
Here, a screen first teaches five rules of chance. Then each of three kinds of chance problem, such as two draws from a bag without putting the first back, is learned from a pair of problems, the kinds in an order drawn from the session's seed. With worked examples (the standard), the first of each pair is shown solved in three steps and the second is solved by the participant; with problem solving (a setting), both are solved. Every learning screen has the same time limit in both conditions, 180 seconds by default, so both groups have the same time with each problem, as Tarmizi and Sweller (1988) gave both their groups a fixed time to study; each solved problem is followed by its answer. Mental effort is rated on the 9-point scale of Paas (1992) after every example and problem. The test has six problems with no time limit and no feedback: three like the practice problems, with new stories and numbers, and three new ones that join two kinds. Answers are typed as percentages. The problems and their worked steps are written for this task in each of the eight languages.
The main score is the percent of the six test problems answered right. The percent over the three similar and the three new problems, the median time per test problem, the mean effort rated while learning and in the test, the percent of practice problems solved and the time spent on the learning screens come with it. Compare groups that learned differently: Paas and Van Merriënboer (1993) combine a group's test scores and effort ratings, as z scores, into the efficiency of its learning condition. Scores support research and do not diagnose.
Scores
- Test Problems Right%
- Percent of the six test problems answered with the right percentage; blank answers count as wrong.
- Higher means more test problems solved
- Similar Problems Right%
- Percent of the three test problems like the practice problems answered right.
- Transfer to problems of a kind already learned
- New Problems Right%
- Percent of the three new test problems, each joining two kinds, answered right.
- Transfer to problems no example showed
- Median Time per Test Problemms
- Median time from a test problem's appearance to its answer, over answered problems, leaving out answers under 100 ms.
- Describes pace; the test has no time limit
- Mental Effort in the Testscore
- Mean mental effort rated after the test problems, from 1 (very, very low) to 9 (very, very high).
- Read with the test score: the same score with less effort means more efficient learning
- Mental Effort While Learningscore
- Mean mental effort rated after the worked examples and practice problems, from 1 to 9.
- Informational
- Practice Problems Right%
- Percent of the problems solved while learning answered right: three with worked examples, six with problem solving.
- Informational
- Time on Learning Screensms
- Total time on the worked examples and practice problems; a screen that ran out counts its whole limit.
- Informational
- Learning Screens Timed Outcount
- Worked examples and practice problems that reached the time limit.
- Informational
- Test Problems Answeredcount
- Test problems submitted with an answer.
- A session with fewer than 3 is invalid
- Blank Test Answerscount
- Test problems submitted with an empty box.
- Lower means more problems were attempted
- Answers Under 100 mscount
- Test answers submitted within 100 ms of the problem appearing.
- Expected to be 0
Use and evidence
114 papers from 2021 to 2025 name this task in their title or abstract, down 22% compared with 2016 to 2020. That ranks it 254 of 670 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 22 |
| 2024 | 25 |
| 2023 | 27 |
| 2022 | 26 |
| 2021 | 14 |
| 2020 | 39 |
| 2019 | 34 |
| 2018 | 26 |
| 2017 | 23 |
| 2016 | 25 |
| 2015 | 33 |
| 2014 | 25 |
| 2013 | 22 |
| 2012 | 23 |
| 2011 | 18 |
| 2010 | 27 |
| 2009 | 19 |
| 2008 | 16 |
| 2007 | 13 |
| 2006 | 20 |
| 2005 | 16 |
| 2004 | 16 |
| 2003 | 7 |
| 2002 | 21 |
| 2001 | 11 |
| 2000 | 7 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 1
- Papers on reliability or validity
- 30
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.48.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Problems
- 3 kinds of chance problem written for this task, each solved in 3 steps with 5 rules of chance: two draws without putting back, at least once in two tries, and the same result twice with putting back. Every answer is a whole percentage, and no two answers are the same.
- Learning
- A screen of the 5 rules, then a pair of problems of each kind, the kinds in an order drawn from the session's seed. With worked examples (standard), the first of each pair is shown solved step by step and the second is solved by the participant; with problem solving, both are solved. Each solved problem is followed by its answer.
- Time
- Each learning screen ends after 180 seconds by default, 60 to 600 as a setting, the same in both conditions, with a bar showing the time left. The test has no time limit.
- Effort
- Mental effort rated after every example and problem on a 9-point scale, from 1 (very, very low) to 9 (very, very high), after Paas (1992).
- Test
- 6 problems with no feedback: 3 like the practice problems, with new stories and numbers, then 3 new ones that join two kinds, each group in an order drawn from the seed.
- Answer
- A typed percentage, in a box that opens the number keypad on phones, with a % sign beside it. No calculator.
- Languages
- Each problem and each worked step is written in all eight languages, with its numbers passed in so a translation cannot drop one.
Give each group of a study one learning condition and compare the groups, as Sweller and Cooper (1985) compared groups; one person's score also depends on what they knew before. Paas and Van Merriënboer (1993) combine a group's test scores and effort ratings into the efficiency of its condition. The scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: worked_examples_test from the scored run, one typed percentage for each of the 6 test problems. Typed answers have no chance level, so accuracy is not judged against chance, and test problems have no time limit, so none can be missed. Fewer than 3 answered invalidates; a run short of its 6 problems is flagged. Answers under 200 ms on at least half of 4 or more answered problems invalidate. One identical answer on 4 or more problems is flagged for review. A hidden tab is flagged.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. A score over fewer than half the test problems cannot be compared with a score over all of them, so it invalidates. A low score is the result the test measures and never invalidates. The 6 answers all differ, so one answer typed for most of the problems cannot follow them; it is flagged. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. Problems stay on screen until they are answered, so a hidden tab does not change what was shown, but it can mean a calculator or help was used; it is flagged.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Learning phase | Worked examples (standard) | Worked examples (standard), Problem solving |
| Time per learning screen (seconds) | 180 | 60 to 600 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Worked Examples Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Sweller J, Cooper GA, 1985; Atkinson RK, Derry SJ, Renkl A, Wortham D, 2000), with the standard settings (learning phase: Worked examples; time per learning screen (seconds): 180). The main score was test problems right (percent of the six test problems answered with the right percentage; blank answers count as wrong).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2(1), 59-89. https://doi.org/10.1207/s1532690xci0201_3
- Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181-214. https://doi.org/10.3102/00346543070002181Design rules: example-problem pairs, two problems of each kind.
- Tarmizi, R. A., & Sweller, J. (1988). Guidance during mathematical problem solving. Journal of Educational Psychology, 80(4), 424-436. https://doi.org/10.1037/0022-0663.80.4.424The same study time for both groups.
- Paas, F. G. W. C. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429-434. https://doi.org/10.1037/0022-0663.84.4.429The 9-point mental effort scale.
- Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1993). The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human Factors, 35(4), 737-743. https://doi.org/10.1177/001872089303500412Efficiency of a learning condition.
Source papers
- Sweller J, Cooper GA (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2(1):59-89.Paradigm source, School students
Questions
How long does the Worked Examples Task take?
About 15 to 25 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Test Problems Right, Similar Problems Right, New Problems Right, Median Time per Test Problem, Mental Effort in the Test, Mental Effort While Learning, Practice Problems Right, Time on Learning Screens, Learning Screens Timed Out, Test Problems Answered, Blank Test Answers and Answers Under 100 ms. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Worked Examples Task used in research?
114 papers from 2021 to 2025 name it in their title or abstract, down 22% compared with 2016 to 2020. That ranks it 254 of 670 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.48.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Worked Examples Task have?
In the published research, 1 paper reports its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Learning phase (Worked examples (standard), Problem solving) and time per learning screen (seconds) (60 to 600). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.