Language
Artificial Grammar Learning
Also called: AGL task, artificial grammar learning task, finite-state grammar learning, implicit grammar learning task
Measure implicit learning: participants copy letter strings made by a hidden grammar, then judge whether new strings follow its rules.
8-11 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 165
- Evidence grade
- D
- Registered clinical studies
- 0
What it measures
Measures learning of the rules behind letter strings without being told them. Participants copy 16 strings of letters from memory, then learn that the strings followed rules and judge 48 new strings as following them or not. The main score is the percent of new strings judged correctly.
Constructs
- Implicit learning
- Rule learning
- Sequence learning
- Statistical learning of fragments
Research uses
- Implicit learning research at the group level
- Language learning and dyslexia research
- Memory systems research, for example in amnesia
How the paradigm works
Artificial grammar learning shows how people pick up the rules behind a set of items without being told them. In the method Reber (1967) introduced, people first study letter strings made by a finite-state grammar, a set of rules for which letter may follow which, and only afterwards learn that rules existed. They then judge new strings as following the rules or not, and usually do better than chance (Nardacchione, Zoccolotti and Marinelli, 2026). The judgments rest both on the rules and on familiar fragments of the strings, and people with amnesia classify new strings as well as others do (Knowlton and Squire, 1996). It is one of the most used methods for studying implicit learning and how rules, similarity and learned fragments contribute (Pothos, 2007), and it is used in research on dyslexia (Nardacchione, Zoccolotti and Marinelli, 2026).
More about the paradigm
This version uses a grammar written by ConductScience, not Reber's or any published one, with the letters D, G, K, L and R. In training, each of 16 strings of 4 to 8 letters is shown for 3 seconds (2 to 5 in the settings) and then copied from memory by typing or clicking the letters, with the right string shown after each copy. The 16 strings come twice (1 to 3 rounds in the settings), in a new order each round, and nothing is said about rules. Participants then learn that the strings followed rules and judge 48 new strings, one at a time, with the F and J keys: 24 follow the grammar and 24 break it at one letter. In each half, 12 strings are made of fragments that were frequent in training and 12 of rare ones, matched across the halves, so knowledge of the rules can be told apart from familiarity with fragments (Knowlton and Squire, 1996). There is no feedback and no time limit. Two practice copies of digit strings come first, and which key means "follows the rules" is drawn for each session.
The main score is the percent of the 48 strings judged correctly; 50% is chance. The task also reports d-prime and criterion c, the share of grammatical and of ungrammatical strings judged to follow the rules, the chunk strength effect (the share of strings made of frequent fragments judged to follow the rules minus the share made of rare fragments), how accurately the training strings were copied, the median judgment time and anticipations (judgments under 100 ms, left out of the scores). Scores support research and do not diagnose.
Scores
- Percent Correct%
- Percent of the 48 test strings judged correctly: follows the rules for grammatical strings, breaks them for the others. Judgments under 100 ms are left out.
- Higher means more of the grammar was learned; 50% is chance
- d-Primed'
- Z(share of grammatical strings judged to follow the rules) minus Z(share of ungrammatical strings judged so), log-linear corrected.
- Higher is better; 0 is chance
- Criterion cz
- Minus the mean of the two z scores.
- Positive values mean a leaning toward breaks the rules
- Grammatical Strings Accepted%
- Share of grammatical strings judged to follow the rules.
- Higher is better
- Ungrammatical Strings Accepted%
- Share of ungrammatical strings judged to follow the rules.
- Lower is better
- Chunk Strength Effect%
- Share of strings made of frequent training fragments judged to follow the rules, minus the share made of rare fragments, over both kinds.
- Positive values mean judgments followed familiar fragments; not better or worse
- Copies Exact%
- Share of training strings copied exactly.
- Quality check
- Letters Copied in Place%
- Mean share of each training string's letters copied in the right place.
- Quality check
- Median Judgment Timems
- Median time of the judgments, leaving out those under 100 ms.
- Lower is faster
- Anticipationscount
- Judgments under 100 ms, left out of every score.
- Quality check
- Judgments Scoredcount
- Test strings with a judgment of 100 ms or more.
- Informational
Use and evidence
165 papers from 2021 to 2025 name this task in their title or abstract, down 28% compared with 2016 to 2020. That ranks it 143 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 55 |
| 2024 | 29 |
| 2023 | 16 |
| 2022 | 34 |
| 2021 | 31 |
| 2020 | 40 |
| 2019 | 52 |
| 2018 | 42 |
| 2017 | 47 |
| 2016 | 47 |
| 2015 | 40 |
| 2014 | 50 |
| 2013 | 43 |
| 2012 | 57 |
| 2011 | 37 |
| 2010 | 53 |
| 2009 | 57 |
| 2008 | 35 |
| 2007 | 23 |
| 2006 | 21 |
| 2005 | 28 |
| 2004 | 24 |
| 2003 | 29 |
| 2002 | 14 |
| 2001 | 14 |
| 2000 | 19 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 1
- Papers on reliability or validity
- 22
- Registered clinical studies using it as an outcome
- 0
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Grammar
- Our own finite-state grammar: 6 states and 14 transitions over the letters D, G, K, L and R. It is not Reber's or any published grammar.
- Training
- 16 strings of 4 to 8 letters. Each is shown for 3 seconds (standard; 2 to 5), then a 500 ms blank, then copied from memory by typing or clicking the letters, and the right string is shown for 1,500 ms. 2 rounds (standard; 1 to 3), in a new order each round. Nothing is said about rules.
- Test
- 48 new strings, each shown until a key press: 24 that follow the grammar and 24 that break it at one letter inside the string. No more than 3 of either kind in a row, with a 500 ms blank between strings. No feedback and no time limit.
- Chunk strength
- In each kind, 12 strings made of fragments that were frequent in training and 12 made of rare ones, matched across the kinds, with the same lengths in all four groups.
- Keys
- F and J. Which key means the string follows the rules is drawn for each session. A judgment under 100 ms is an anticipation and is left out.
- Practice
- 2 copies of digit strings with feedback, so practice shows nothing of the grammar.
The method follows Reber (1967), and the test crosses grammaticality with chunk strength as in Knowlton and Squire (1996). The grammar and every string are ConductScience's own. When four implicit learning tasks were retested a week apart, artificial grammar learning did not reach medium reliability (Kalra and colleagues, 2019), so compare groups rather than individuals. The scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: artificial_grammar_test from the scored run, one key press per string. Accuracy significantly below chance (one in two) invalidates and accuracy not above chance is flagged. Judgments under 200 ms on at least half of the strings invalidate. The same key on 90% of at least 12 judgments, with accuracy not above chance, invalidates. Fewer than half the letters copied in place during training, or no training copies, is flagged and withholds the percentile. A hidden tab withholds the percentile.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. Learning a grammar can be slight, so chance-level judgments are only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). Half the test strings follow the grammar and half break it, so a key pressed for nearly every string cannot follow the strings. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. The judgments rest on what was copied, so a training phase in which most letters were not copied in place leaves them uninterpretable. A hidden tab lets the study, delay or response timers run without the participant, so a percentile is withheld after one.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Training rounds | 2 | 1 to 3 |
| Time each string is shown (seconds) | 3 | 2 to 5 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops and laptops. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Artificial Grammar Learning task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Reber, 1967; Knowlton & Squire, 1996), with the standard settings (training rounds: 2; time each string is shown (seconds): 3). The main score was percent correct (percent of the 48 test strings judged correctly: follows the rules for grammatical strings, breaks them for the others).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Reber, A. S. (1967). Implicit learning of artificial grammars. Journal of Verbal Learning and Verbal Behavior, 6(6), 855-863. https://doi.org/10.1016/S0022-5371(67)80149-X
- Knowlton, B. J., & Squire, L. R. (1996). Artificial grammar learning depends on implicit acquisition of both abstract and exemplar-specific information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(1), 169-181. https://doi.org/10.1037/0278-7393.22.1.169Grammaticality crossed with chunk strength, and the chunk strength measure.
- Nardacchione, G., Zoccolotti, P., & Marinelli, C. V. (2026). Procedural or declarative deficit in adults with developmental dyslexia? A study of artificial grammar learning. PLOS ONE, 21(7), e0352337. https://doi.org/10.1371/journal.pone.0352337Copying strings shown for 3 seconds, with feedback, and violations made by changing one element.
- Pothos, E. M. (2007). Theories of artificial grammar learning. Psychological Bulletin, 133(2), 227-244. https://doi.org/10.1037/0033-2909.133.2.227Review.
- Kalra, P. B., Gabrieli, J. D. E., & Finn, A. S. (2019). Evidence of stable individual differences in implicit learning. Cognition, 190, 199-211. https://doi.org/10.1016/j.cognition.2019.05.007How stable the scores are.
Source papers
- Knowlton BJ, Squire LR (1996). Artificial grammar learning depends on implicit acquisition of both abstract and exemplar-specific information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(1):169-181.Paradigm source, Adults
- Kalra PB, Gabrieli JDE, Finn AS (2019). Evidence of stable individual differences in implicit learning. Cognition, 190:199-211.Paradigm source, Young adults
- ConductCognition platform normsParadigm source, N/A
Questions
How long does the Artificial Grammar Learning take?
About 8 to 11 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops and laptops. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Percent Correct, d-Prime, Criterion c, Grammatical Strings Accepted, Ungrammatical Strings Accepted, Chunk Strength Effect, Copies Exact, Letters Copied in Place, Median Judgment Time, Anticipations and Judgments Scored. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Artificial Grammar Learning used in research?
165 papers from 2021 to 2025 name it in their title or abstract, down 28% compared with 2016 to 2020. That ranks it 143 of 395 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Artificial Grammar Learning have?
In the published research, 1 paper reports its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Which settings can a researcher change?
Training rounds (1 to 3) and time each string is shown (seconds) (2 to 5). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.