Language
Incidental Vocabulary Task
Also called: incidental vocabulary learning, incidental word learning, learning words from context, incidental vocabulary acquisition
Measure how many new words people learn while reading for meaning. Participants read short stories with made-up words, then take a surprise test of each word's form and meaning, scored by how often the word appeared.
8-11 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 345
- Evidence grade
- D
- Registered clinical studies
- 0
On her back Lena carried a rumepo, a basket woven from reeds.
What it measures
Measures how many new words people pick up while reading for meaning. Participants read four short stories, each with three made-up words whose meaning the story makes clear, and answer a question about each story. A surprise test then asks which of four words was in the stories and what each word means. Words appear six times, three times or once, so scores show how learning grows with each encounter.
Constructs
- Incidental word learning
- Vocabulary acquisition
- Reading for meaning
Research uses
- Reading research
- Second-language research
- Vocabulary research
How the paradigm works
Readers pick up new words from context, a little at each encounter. In this incidental vocabulary task, participants read four short stories for meaning and answer one question about each story's events. Each story holds three made-up words for concrete things, such as a fruit, a rope or a warm wind, and the story makes each meaning clear: one word appears six times, one three times and one once. The instructions mention only the stories. A surprise test then shows four words, the taught word, a look-alike that changes one letter, an untaught made-up word and its look-alike, and asks which one was in the stories; a second part shows each word with four meanings from the stories. The stories, questions and meanings are written in each of the eight languages, and the made-up words are the same in every language, in katakana in Japanese. A setting adds a note under each story that gives its words' meanings, as a glossed text does.
Scores
- Meanings Chosen%
- Percentage of answered meaning questions on which the word's own meaning was chosen.
- Higher means more word meanings learned
- Word Forms Recognized%
- Percentage of answered form questions on which the word from the stories was chosen over its look-alike and two untaught words.
- Higher means more word forms learned
- Meanings, Six Encounters%
- Meanings chosen correctly for the four words met six times.
- Higher means more learned
- Meanings, Three Encounters%
- Meanings chosen correctly for the four words met three times.
- Higher means more learned
- Meanings, One Encounter%
- Meanings chosen correctly for the four words met once.
- Higher means more learned
- Forms, Six Encounters%
- Word forms recognized for the four words met six times.
- Higher means more learned
- Forms, Three Encounters%
- Word forms recognized for the four words met three times.
- Higher means more learned
- Forms, One Encounter%
- Word forms recognized for the four words met once.
- Higher means more learned
- Meaning Gain, One to Six Encounterspoints
- Meanings chosen correctly at six encounters minus at one encounter, in percentage points.
- Higher means more learning from repeated encounters
- Form Gain, One to Six Encounterspoints
- Word forms recognized at six encounters minus at one encounter, in percentage points.
- Higher means more learning from repeated encounters
- Story Questions Correct%
- Percentage of the story questions answered correctly.
- Under 50 is flagged: the stories may not have been read for meaning
- Median Reading Timeseconds
- Median time a story stayed on screen.
- Informational; under 10 seconds is flagged
- Answers Under 100 mscount
- Answers made within 100 ms of a question appearing.
- 0 expected
- Word Questions Answeredcount
- Form and meaning questions answered in the most complete run.
- 24 standard
- Word Questions Plannedcount
- Form and meaning questions planned: two for each of the 12 words.
- 24 standard
Use and evidence
345 papers from 2021 to 2025 name this task in their title or abstract, up 26% compared with 2016 to 2020. That ranks it 87 of 632 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 90 |
| 2024 | 77 |
| 2023 | 70 |
| 2022 | 60 |
| 2021 | 48 |
| 2020 | 67 |
| 2019 | 50 |
| 2018 | 56 |
| 2017 | 51 |
| 2016 | 50 |
| 2015 | 46 |
| 2014 | 62 |
| 2013 | 50 |
| 2012 | 61 |
| 2011 | 54 |
| 2010 | 42 |
| 2009 | 38 |
| 2008 | 30 |
| 2007 | 21 |
| 2006 | 14 |
| 2005 | 6 |
| 2004 | 18 |
| 2003 | 10 |
| 2002 | 10 |
| 2001 | 7 |
| 2000 | 9 |
Use grade B
300 to 999 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 23
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes classroom and media studies of second-language learners.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.44.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Stories
- 4 original stories in each of 8 languages, read at the participant's own pace, each followed by 1 question about its events.
- Made-up words
- 3 in each story, 12 in all: in each story, 1 word appears 6 times, 1 appears 3 times and 1 appears once. The words are the same in every language, in katakana in Japanese.
- Surprise test
- The instructions mention only the stories. Form: 4 words, the taught word, a look-alike that changes 1 letter, an untaught word and its look-alike. Meaning: each word with 4 meanings from the stories.
- Word notes
- A setting adds a note under each story that gives its words' meanings, as a glossed text does. Off by default.
- Order
- Seeded story and question order. The right answer is in each of the 4 positions equally often.
- Practice
- 1 short text and question with feedback before the stories.
- Scores
- Percentage of meanings chosen correctly and of word forms recognized, for all words and by number of encounters, with the gain from 1 to 6 encounters. Answers under 100 ms count as anticipations.
The stories and made-up words are our own. The test reveals its words, so each person takes it once. Scores support research and do not diagnose. No percentile output.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: incidental_vocabulary_form and incidental_vocabulary_meaning from the most complete run, each checked against the canonical forms and meanings for its word and language; one of four options per question. Accuracy significantly below chance (one in four) invalidates and accuracy not above chance is flagged. Answers under 200 ms on at least half of the questions invalidate. The same position on 90% of at least 12 answers, with accuracy not above chance, invalidates. Fewer than half of the planned questions invalidates and any shortfall is flagged. Fewer than half of the story questions right, or a median under 10 seconds on a story, is flagged.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). The right option is in each position equally often. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. The story questions check that the stories were read for meaning, which incidental learning needs, so missing them is flagged rather than invalidating. The words are made up, so nothing can be looked up in another tab.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Word notes | No notes (standard) | No notes (standard), Notes under each story |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Incidental Vocabulary Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Nagy WE, Herman PA, Anderson RC, 1985; Hulstijn JH, Laufer B, 2001), with the standard settings (word notes: No notes). The main score was meanings chosen (percentage of answered meaning questions on which the word's own meaning was chosen).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Nagy, W. E., Herman, P. A., & Anderson, R. C. (1985). Learning words from context. Reading Research Quarterly, 20(2), 233–253. https://doi.org/10.2307/747758Readers met unfamiliar words in natural text, then chose their meanings in multiple-choice items that used other target words' meanings as wrong answers.
- Hulstijn, J. H., & Laufer, B. (2001). Some empirical evidence for the involvement load hypothesis in vocabulary acquisition. Language Learning, 51(3), 539–558. https://doi.org/10.1111/0023-8333.00164Retention of ten unfamiliar words after reading for comprehension, compared with tasks of greater involvement.
- Hulstijn, J. H., Hollander, M., & Greidanus, T. (1996). Incidental vocabulary learning by advanced foreign language students: The influence of marginal glosses, dictionary use, and reoccurrence of unknown words. The Modern Language Journal, 80(3), 327–339. https://doi.org/10.1111/j.1540-4781.1996.tb01614.xMarginal glosses and words met once or three times in a story; the word notes setting and the scores by encounters follow this design.
- Pellicer-Sánchez, A. (2016). Incidental L2 vocabulary acquisition from and while reading: An eye-tracking study. Studies in Second Language Acquisition, 38(1), 97–130. https://doi.org/10.1017/S0272263115000224Made-up words in a story read for comprehension, followed by form recognition and then meaning recognition tests.
Source papers
- Nagy, Herman and Anderson (1985); Hulstijn, Hollander and Greidanus (1996): learning words from reading.Paradigm source, Adults who read the session language
Questions
How long does the Incidental Vocabulary Task take?
About 8 to 11 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Meanings Chosen, Word Forms Recognized, Meanings, Six Encounters, Meanings, Three Encounters, Meanings, One Encounter, Forms, Six Encounters, Forms, Three Encounters, Forms, One Encounter, Meaning Gain, One to Six Encounters, Form Gain, One to Six Encounters, Story Questions Correct, Median Reading Time, Answers Under 100 ms, Word Questions Answered and Word Questions Planned. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Incidental Vocabulary Task used in research?
345 papers from 2021 to 2025 name it in their title or abstract, up 26% compared with 2016 to 2020. That ranks it 87 of 632 test families in the ConductCognition test census (use grade B). Source: ConductCognition test census 0.44.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Incidental Vocabulary Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Word notes (No notes (standard), Notes under each story). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.