Language
Grammaticality Judgment Task
Also called: GJT, acceptability judgment task, grammaticality judgement task
Measure sensitivity to grammatical errors with matched correct and incorrect English sentences in six error types.
4-6 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 1,140
- Evidence grade
- C
- Registered clinical studies
- 4
What it measures
Measures sensitivity to grammatical errors in English. Participants read 60 sentences one at a time and press one key when a sentence is correct and another when it has an error. Half the sentences have one error in agreement, tense, plural, word order, verb structure or pronoun case.
Constructs
- Grammatical knowledge
- Syntax and morphology
- Language comprehension
Research uses
- Second-language and bilingualism research
- Language development and aphasia research
- Within-study group comparison
How the paradigm works
The grammaticality judgment task measures how well people tell correct English sentences from sentences with a grammatical error. Participants read one sentence at a time and press one key if it is correct English and the other key if it has an error. Judgments like these are a standard measure of grammatical knowledge in second-language research (Johnson and Newport 1989) and aphasia research (Linebarger, Schwartz and Saffran 1983).
More about the paradigm
The 60 sentence frames were written for this task; no published sentence set is used. Each frame has a correct version and a version with one error, in six error types: subject-verb agreement, verb tense or form, noun plural, word order, verb argument structure and pronoun case. Two lists, drawn from the session's random seed, show every participant each frame once: 30 correct sentences and 30 with an error, 5 and 5 of each type, in a seeded order with no more than 3 correct or 3 incorrect sentences in a row. Which key means correct is also drawn from the seed and recorded. Sentences stay on screen until a key press, with no time limit, after 4 practice sentences with feedback. The sentences are English.
The main score is d-prime: how well judgments separate correct sentences from sentences with an error, from signal detection theory with the log-linear correction, so 0 is chance. The response criterion c, accuracy, hit and false-alarm rates, accuracy for each error type and the median time of correct judgments are also reported. Scores support research and do not diagnose.
Scores
- d-Primed'
- Z(hit rate) minus Z(false-alarm rate), log-linear corrected, where a hit is an incorrect sentence judged to have an error.
- Higher is better; 0 is chance
- Criterion cz
- Minus the mean of the two z scores.
- Positive values mean a tendency to accept sentences as correct
- Accuracy%
- Share of sentences judged correctly.
- Higher is better
- Hit Rate%
- Share of sentences with an error that were judged to have one.
- Higher is better
- False-Alarm Rate%
- Share of correct sentences judged to have an error.
- Lower is better
- Accuracy, Agreement%
- Share of subject-verb agreement sentences judged correctly, both versions.
- Higher is better
- Accuracy, Tense%
- Share of verb tense or form sentences judged correctly, both versions.
- Higher is better
- Accuracy, Plural%
- Share of noun plural sentences judged correctly, both versions.
- Higher is better
- Accuracy, Word Order%
- Share of word order sentences judged correctly, both versions.
- Higher is better
- Accuracy, Argument Structure%
- Share of verb argument structure sentences judged correctly, both versions.
- Higher is better
- Accuracy, Pronoun Case%
- Share of pronoun case sentences judged correctly, both versions.
- Higher is better
- Median Correct Timems
- Median time of correct judgments; answers under 100 ms are left out.
- Lower is faster
Use and evidence
1,140 papers from 2021 to 2025 name this task in their title or abstract, up 4% compared with 2016 to 2020. That ranks it 21 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 242 |
| 2024 | 245 |
| 2023 | 212 |
| 2022 | 203 |
| 2021 | 238 |
| 2020 | 241 |
| 2019 | 221 |
| 2018 | 212 |
| 2017 | 203 |
| 2016 | 224 |
| 2015 | 192 |
| 2014 | 158 |
| 2013 | 164 |
| 2012 | 161 |
| 2011 | 127 |
| 2010 | 133 |
| 2009 | 90 |
| 2008 | 89 |
| 2007 | 80 |
| 2006 | 77 |
| 2005 | 49 |
| 2004 | 52 |
| 2003 | 50 |
| 2002 | 35 |
| 2001 | 24 |
| 2000 | 35 |
Use grade A
1,000 or more papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 4
- Papers on reliability or validity
- 166
- Registered clinical studies using it as an outcome
- 4
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Sentences
- 60 sentence frames, each written correct and with one error, in six error types of 10: subject-verb agreement, verb tense or form, noun plural, word order, verb argument structure and pronoun case.
- Lists
- Two lists drawn from the session's seed show each frame once: 30 correct sentences and 30 with an error, 5 and 5 of each type.
- Order
- Drawn from the session's seed, with no more than 3 correct or 3 incorrect sentences in a row and no two of the same error type in a row.
- Keys
- F and J; which key means correct is drawn from the session's seed and saved.
- Time
- Each sentence stays until a key press, with no time limit, and a 500 ms blank screen comes between sentences.
- Practice
- 4 sentences with feedback for 1,500 ms.
- Score
- d-prime with the log-linear correction, treating a sentence with an error as the signal.
The sentences were written for this task; no published sentence set is used. The sentences are English.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: grammaticality_judgment_test from the scored run, one key press per sentence. Accuracy significantly below chance (one in two) invalidates and accuracy not above chance is flagged. Judgments under 200 ms on at least half of the sentences invalidate. The same key on 90% of at least 12 judgments, with accuracy not above chance, invalidates. A session run in another language is flagged. A hidden tab is flagged.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). Half the sentences are correct and half have an error, so a key pressed for nearly every sentence cannot follow the sentences. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. The word lists are English. A session run in another language shows English words to a participant who may not read English fluently, so it is flagged for review. Sentences stay on screen until a key press, so a hidden tab does not change what was shown; it is flagged.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Sentences per error type | 10 | 6 to 10 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops and laptops. Participant devices
- Instructions and screens: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. Words and sentences: English.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Grammaticality Judgment Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Johnson & Newport, 1989), with the standard settings (sentences per error type: 10). The main score was d-prime (Z(hit rate) minus Z(false-alarm rate), log-linear corrected, where a hit is an incorrect sentence judged to have an error).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 21(1), 60-99. https://doi.org/10.1016/0010-0285(89)90003-0
- Linebarger, M. C., Schwartz, M. F., & Saffran, E. M. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13(3), 361-392. https://doi.org/10.1016/0010-0277(83)90015-X
Source papers
- Johnson JS, Newport EL (1989). Critical period effects in second language learning: the influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 21(1):60-99.Paradigm source, Adults
- Linebarger MC, Schwartz MF, Saffran EM (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13(3):361-392.Paradigm source, Adults
Questions
How long does the Grammaticality Judgment Task take?
About 4 to 6 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops and laptops. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
d-Prime, Criterion c, Accuracy, Hit Rate, False-Alarm Rate, Accuracy, Agreement, Accuracy, Tense, Accuracy, Plural, Accuracy, Word Order, Accuracy, Argument Structure, Accuracy, Pronoun Case and Median Correct Time. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Grammaticality Judgment Task used in research?
1,140 papers from 2021 to 2025 name it in their title or abstract, up 4% compared with 2016 to 2020. That ranks it 21 of 395 test families in the ConductCognition test census (use grade A). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Grammaticality Judgment Task have?
In the published research, 4 papers report its test-retest reliability and 4 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Which settings can a researcher change?
Sentences per error type (6 to 10). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
The instructions and screens are in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. The words and sentences stay in English, so sessions in other languages are flagged for review. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.