ConductCognition
  • Domains

    Tests

    • Processing speed55
    • Attention143
    • Working memory51
    • Executive function191
    • Episodic memory84
    • Response inhibition10
    • Social cognition97
    • Language50

    Most used

    Evidence

    1. Verbal Fluency TestGradeA
    2. Stroop Color-Word TestGradeA
    3. Trail Making TestGradeA
    4. N-Back TaskGradeA
    5. Go/No-Go TaskGradeA
    6. Digit Span TestGradeA
    7. Symbol Coding TestGradeA
    8. Verbal Learning TestGradeA
    REDTry it nowStroop Color-Word Test
    All 681 tests

    Browse by

    • Phone380
    • Tablet415
    • Untimed419
    • Up to 5 minutes81
    • Evidence grade A48
  • First studies

    Grade, length

    • Reaction Time StudyGradeA7 min
    • Pilot ScreeningGradeA15 min
    • Medication StudiesGradeA−26 min
    • Language-Minimized ScreeningGradeA−30 min

    Age groups

    • Older Adult Executive SpeedGradeA20 min
    • School-AgeGradeA−33 min

    Memory

    • At-Home MemoryGradeA26 min
    • Working Memory and LearningGradeA−19 min
    • Associative MemoryGradeB32 min
    • Working Memory CapacityGradeB34 min

    Language and perception

    • LanguageGradeA−33 min
    • Visual PerceptionGradeB27 min
    • Listening and Auditory ProcessingGradeB31 min

    Executive function and attention

    • Executive ControlGradeA18 min
    • Attention and ImpulsivityGradeA−26 min
    • Visual AttentionGradeB27 min
    • Creativity and Flexible ThinkingGradeB30 min
    • Reasoning and SpatialGradeB32 min

    Movement

    • Motor and AttentionGradeA−29 min
    • Motor LearningGradeB34 min

    Behavior, mood and choice

    • Food Choice and Self-ControlGradeA−29 min
    • Sleep and FatigueGradeB23 min
    • Social CognitionGradeB23 min
    • Impulsivity and Substance UseGradeB26 min
    • Emotion and MoodGradeB28 min
    • Decisions and RewardGradeB29 min
    All 26 batteries

    Find one

    Up to 15 minutes2Up to 30 minutes19Phone8
    Test order labBattery gradesHow batteries work
  • Use and evidence

    • Overview
    • Compare all 681 tests
    • Test-retest
    • Reliability and validity
    • Battery grades26
    • Download and cite

    Clinical trials

    • Overview
    • By condition25
    • By test103
    • Trial design benchmark
    • Cognitive outcomes, 2026

    Conditions

    Registered studies

    • Dementia and Alzheimer's1,673
    • Aging, frailty and exercise1,400
    • Depression and bipolar disorder1,249
    • Stroke1,237
    • Substance use and addiction904
    • Parkinson's disease894
    • Cancer762
    All 25 conditions
    16,526Registered studies name a cognitive test in an outcome measure11,848 name a test you can run hereExplore the studies
  • 1

    Build

    • Study design
    • Customize tests
    • Repeat visits
    • Study cost calculator
    2

    Recruit

    • Recruitment setup
    • Prolific
    • Sona Systems
    • CloudResearch Connect
    • Qualtrics
    3

    Run

    • Remote testing
    • Participant devices
    • Phone timing check
    • Offline edition
    4

    Results

    • Automated scoring
    • Test norms
    • Study Quality Monitor
    • Data export
    Platform overview

    Compliance

    • HIPAA-ready testing
    • US clinical research
    • Data Policy
    • Privacy Policy
    • Terms of Use
  • Research areas

    Tests used in each

    • Aging and dementia202
    • Decisions and reward136
    • Child development and education114
    • Social behavior87
    • Language and bilingualism80
    • Psychiatric groups35
    • Training and intervention31
    • Mood, anxiety and trauma28
    • Substance use and impulsivity26
    • Neurological and movement research21
    • Sleep and fatigue17
    • Brain injury and concussion14
    • ADHD11
    • Eating and obesity11
    • Medication and drug trials8

    Research settings

    • Student participant pools
    • Teaching a class
    • Clinical trials
    • US clinical research
    • Sites without internet
    Planning a study?
  • Methods

    • How evidence grades work
    • How test batteries work
    • How test order works
    Methods and questions

    Help

    • FAQ
    • Recruitment guides
    • Compare platforms

    Updates

    • Blog
    • New tests by email
    Latest post10 November 2026What you can do in ConductCognition todayA short tour of ConductCognition as it stands on 10 November. After this, we'll post when something changes how you run a study.Read the post
  • Pricing
Start Free
Log In
Log InStart Free
  1. Home
  2. /Tests
  3. /Human or AI Text Task

Reasoning

Human or AI Text Task

Also called: AI text detection task, human or machine text task, Turing test for text, machine-generated text detection by people, real or fake text, AI authorship judgment task

Measure how well people tell texts written by people from texts written by an AI system: participants read short answers to everyday questions, say who wrote each and how sure they are, and see at the end which were which.

8-11 minUntimed

Create a study with this test

Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.

Papers, 2021 to 2025
84
Evidence grade
D
Registered clinical studies
0
How these are counted
Why do geese fly in a V?
Each bird flies a little above the one in front, which cuts wind resistance, and the birds take turns at the front.
Who wrote this answer?
A personAn AI system
d primeAccuracy, bias and confidence
Read a short answer, say whether a person or an AI system wrote it, and how sure you are
  • Overview
  • Evidence
  • Task design
  • Quality checks
  • Settings
  • Cite
  • Questions

What it measures

Measures how well people tell texts written by people from texts written by an AI system. Participants read 24 short English answers to everyday questions, half written by people and half by an AI system with the same topics and lengths, choose who wrote each one and rate how sure they are, from 1 to 5. Scores are accuracy, sensitivity and bias toward either answer. The end says which texts were which.

Constructs

  • Detection of AI-written text
  • Authorship judgment
  • Confidence in one's own judgments
  • Response bias

Research uses

  • Research on how people judge AI-generated text
  • Research on media literacy and trust in generated content
  • Within-study group comparison

How the paradigm works

The human or AI text task measures how well people tell texts written by people from texts written by an AI system. Clark and colleagues (2021) asked non-expert readers to say whether stories, news articles and recipes were written by a person or by GPT-2 or GPT-3, and found that without training they told GPT-3 text from human text at random chance level, with training lifting accuracy to at most 55%. Köbis and Mossink (2021) ran two incentivized experiments with 830 people on poems written by GPT-2 and found that readers failed to reliably detect the algorithm's poems when a person picked the best of its samples. Jakesch, Hancock and Naaman (2023) found that people could not detect AI-generated self-presentations, because they relied on flawed heuristics about what AI text looks like.

More about the paradigm

This task shows 24 short answers to everyday questions, such as why geese fly in a V. Twelve were written by people, the science librarians of the Library of Congress, and 12 by an AI system, Claude Sonnet 5.5, from the same questions and with the same lengths. Each question appears once in a session, in one of its two versions drawn from the session's seed, so across sessions neither topic nor length can tell the two kinds apart. For each text the participant chooses who wrote it and says how sure they are, from 1 to 5. The scores treat the AI texts as the signal: sensitivity (d prime), the response criterion, which shows a lean toward answering a person or an AI system, accuracy for each kind of writer, and how much more sure the participant was when right than when wrong. The end of the session says which texts were which. The scores describe how a reader handles these texts and say nothing about anyone's writing.

Scores

d primed'
Sensitivity to AI-written texts: the corrected rate of AI texts judged AI minus the corrected rate of people's texts judged AI, on the z scale.
Informational. 0 means the two kinds are not told apart; below 0 means people's texts were judged AI more often than AI texts
Response criterionz
Lean toward one answer. Positive values lean toward answering A person; negative values lean toward An AI system.
Informational
Accuracy%
Percent of answered texts judged correctly.
Informational. 50 is chance with equal numbers of the two kinds
Accuracy for people's texts%
Percent of the texts written by people that were judged to be written by a person.
Informational
Accuracy for AI texts%
Percent of the texts written by the AI system that were judged to be written by an AI system.
Informational
Texts judged AI-written%
Percent of all answered texts judged to be written by an AI system, whoever wrote them.
Informational. 50 means no lean
Mean confidencescore
Mean of the confidence ratings, from 1 (just guessing) to 5 (completely sure).
Informational
Confidence when rightscore
Mean confidence on texts judged correctly, from 1 to 5.
Informational
Confidence when wrongscore
Mean confidence on texts judged wrongly, from 1 to 5.
Informational
Confidence gapscore
Confidence when right minus confidence when wrong.
Positive means more sure when right; 0 or below means confidence did not follow accuracy
Median answer timems
Median time from the text appearing to the answer, over answers of 100 ms or more. It includes reading.
Informational
Texts answeredcount
Texts with a recorded answer and a known text.
Quality check
Planned textscount
12 or 24 test texts.
Informational
Answers under 100 mscount
Answers and confidence ratings made less than 100 ms after their screen appeared.
0 expected
Session language0/1
1 when the session ran in another language than English, since the texts are English.
Quality check; 1 is flagged for review

Use and evidence

84 papers from 2021 to 2025 name this task in their title or abstract. That ranks it 321 of 711 test families in our census. See the counts for every test.

2021 to 2025, the years gradedEarlier years
Papers per year naming the Human or AI Text Task, 2000 to 2025Bar chart. 2021: 0; 2022: 0; 2023: 9; 2024: 27; 2025: 48 papers. Peak year 2025.02040602000: 0 papers2001: 0 papers2002: 0 papers2003: 0 papers2004: 0 papers2005: 0 papers2006: 0 papers2007: 0 papers2008: 0 papers2009: 0 papers2010: 0 papers2011: 0 papers2012: 0 papers2013: 0 papers2014: 0 papers2015: 0 papers2016: 0 papers2017: 0 papers2018: 0 papers2019: 0 papers2020: 0 papers2021: 0 papers2022: 0 papers2023: 9 papers2024: 27 papers2025: 48 papers200020052010201520202025
Papers per year naming the Human or AI Text Task, 2000 to 2025Bar chart. 2021: 0; 2022: 0; 2023: 9; 2024: 27; 2025: 48 papers. Peak year 2025.02040602000: 0 papers2001: 0 papers2002: 0 papers2003: 0 papers2004: 0 papers2005: 0 papers2006: 0 papers2007: 0 papers2008: 0 papers2009: 0 papers2010: 0 papers2011: 0 papers2012: 0 papers2013: 0 papers2014: 0 papers2015: 0 papers2016: 0 papers2017: 0 papers2018: 0 papers2019: 0 papers2020: 0 papers2021: 0 papers2022: 0 papers2023: 9 papers2024: 27 papers2025: 48 papers200020052010201520202025
Papers per year whose title or abstract names this task (OpenAlex).
Show the numbers
YearPapers
202548
202427
20239
20220
20210
20200
20190
20180
20170
20160
20150
20140
20130
20120
20110
20100
20090
20080
20070
20060
20050
20040
20030
20020
20010
20000
D

Use grade D

30 to 99 papers from 2021 to 2025.

D

Evidence grade D

Little published measurement evidence found.

Papers on test-retest reliability
0
Papers on reliability or validity
25
Registered clinical studies using it as an outcome
0

Measured on ConductCognition

Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.

No pooled figure yet.

Note on the counts: includes studies that compare texts by software or by their wording.

How the grades work

Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.

Source: ConductCognition test census 0.58.0, counted 16 November 2026 from OpenAlex and ClinicalTrials.gov.

Task design

The trial sequence as the test runs in ConductCognition.

Texts
24 everyday questions, such as why geese fly in a V. Each has two answers of the same length, one written by people and one by an AI system. A session shows each question once, in one of its two versions drawn from the session's seed: 12 written by people and 12 by the AI system, with no more than 3 of one kind in a row. They are English in every session.
People's answers
Answers of the Library of Congress's Everyday Mysteries, written by its science librarians and posted from 2018 to 2020. Each was checked against copies of the Library's page made before June 2022, before AI systems wrote text at this level. They are works of the United States Government, in the public domain. They are shown as published, apart from joining paragraphs, stopping at a sentence end and leaving out citations.
AI answers
Written by Claude Sonnet 5.5 from the question and a word target equal to the length of the people's answer, under its maker's commercial terms. No wording is taken from the people's answers, and the two lengths agree within 10 percent.
Trial
The question, the text in a light box and two buttons, A person and An AI system, with their sides drawn from the seed. Then how sure the participant is, from 1 (just guessing) to 5 (completely sure), on five buttons. No time limit.
Session
2 practice texts with feedback, then 24 with none, or 12 in the short form. The end of the session says which texts were written by people.
Scores
Sensitivity (d prime, with the AI texts as the signal), accuracy, accuracy for each kind of writer, the share of texts judged AI-written, the response criterion, mean confidence, and how much more sure people were when right than when wrong.

Clark and colleagues (2021) found non-expert readers at chance in telling GPT-3 text from human text in stories, news articles and recipes, and Jakesch, Hancock and Naaman (2023) found readers guided by flawed heuristics. This task gives every reader the same questions in both versions and reports sensitivity, bias and confidence. The texts are short answers on everyday science, not stories or profiles, so the scores describe how readers handle these texts and are best compared between groups. They say nothing about anyone's writing.

Quality checks

Each session is marked valid, needs review or invalid, with the reason, before you see its scores.

Scored rows: human_or_ai_text_test from the scored run. Below-chance accuracy, one-sided answers and confident wrong answers are retained, because how people read the texts is the finding. Answers under 1.5 seconds on at least half the texts invalidate. An incomplete run is flagged for review, and so is a session that ran in another language.

Why these rules

Studies of human and machine text find readers near chance (Clark et al., 2021; Jakesch et al., 2023), so chance, bias and misplaced confidence are outcomes, not exclusion rules. An answer under 1.5 seconds cannot follow reading a text of 60 words or more, so the session is invalid only when at least half the answers are that fast (Wise and Kong 2005). The word lists are English. A session run in another language shows English words to a participant who may not read English fluently, so it is flagged for review.

No percentile. Scores and quality flags only.

Running this test

Settings you can change

SettingStandardAllowed
Texts of each kind12 (24 texts, standard)12 (24 texts, standard), 6 (12 texts)

Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.

How test settings work

Need a setting this test does not have?

Devices and languages

  • Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
  • Instructions and screens: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. Words and sentences: English.
  • Available on: Every plan, including the free plan.
  • No percentile. Scores and quality flags only. How norms work
  • Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.

Cite and describe

Methods text, with the standard settings

Participants completed the Human or AI Text Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Clark et al., 2021; Jakesch, Hancock and Naaman, 2023; Köbis and Mossink, 2021), with the standard settings (texts of each kind: 12 (24 texts, standard)). The main score was d prime (sensitivity to AI-written texts: the corrected rate of AI texts judged AI minus the corrected rate of people's texts judged AI, on the z scale).

ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.

  • Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021). All that's "human" is not gold: Evaluating human evaluation of generated text. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, 7282-7296. https://doi.org/10.18653/v1/2021.acl-long.565The method: non-experts judge whether short texts were written by a person or an AI system; without training they did no better than chance for GPT-3 text, and training raised accuracy to at most 55%.
  • Jakesch, M., Hancock, J. T., & Naaman, M. (2023). Human heuristics for AI-generated language are flawed. Proceedings of the National Academy of Sciences, 120(11), e2208839120. https://doi.org/10.1073/pnas.2208839120People could not detect AI-generated self-presentations, because their judgments rested on flawed heuristics about AI language.
  • Köbis, N., & Mossink, L. D. (2021). Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in Human Behavior, 114, 106553. https://doi.org/10.1016/j.chb.2020.106553An incentivized version of the Turing test, with 830 people in two experiments, in which readers failed to reliably detect poems the algorithm produced when a person chose its best sample.

Source papers

    Questions

    How long does the Human or AI Text Task take?

    About 8 to 11 minutes, as listed in the test library.

    What device does a participant need?

    Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.

    What scores does it produce?

    d prime, Response criterion, Accuracy, Accuracy for people's texts, Accuracy for AI texts, Texts judged AI-written, Mean confidence, Confidence when right, Confidence when wrong, Confidence gap, Median answer time, Texts answered, Planned texts, Answers under 100 ms and Session language. Each session is also marked valid, needs review or invalid, with the reason.

    How widely is the Human or AI Text Task used in research?

    84 papers from 2021 to 2025 name it in their title or abstract. That ranks it 321 of 711 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.58.0, counted 16 November 2026 from OpenAlex and ClinicalTrials.gov.

    How much reliability evidence does the Human or AI Text Task have?

    In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).

    Does it report a percentile?

    No. Scores and quality flags only.

    Which settings can a researcher change?

    Texts of each kind (12 (24 texts, standard), 6 (12 texts)). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.

    Which languages can participants use?

    The instructions and screens are in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. The words and sentences stay in English, so sessions in other languages are flagged for review. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.

    Is this a clinical test?

    No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.

    Every test: the full test list.

    New tests and features, about once a month. Unsubscribe at any time.

    Run this test in your next study

    Start free with 10 free participants on any test, then use participant credits or a yearly plan.

    Start free

    Related tests

    Real or AI Face TaskHeadline Truth TaskProfound Statements TaskPerceptual Confidence Task
    ConductCognition

    Hosted browser-native cognitive testing for academic and commercial research workflows.

    A product of ConductScience, Inc.

    Tests

    • All cognitive tests
    • Test batteries
    • Use and evidence
    • Test norms
    • Trail Making Test
    • N-Back Task
    • Big Five personality test
    • Automated scoring
    • Study design
    • New tests by email

    Guides

    • Methods and questions
    • Blog
    • Recruitment setup
    • Participant devices
    • Prolific
    • Sona Systems
    • CloudResearch Connect
    • Qualtrics

    Use cases

    • Student participant pools
    • Aging and dementia research
    • Clinical trials
    • US clinical research
    • Remote testing

    Compare and buy

    • Pricing
    • Compare platforms
    • Offline edition
    • FAQ

    Trust

    • Study Quality Monitor
    • HIPAA-ready testing
    • Data export
    • Privacy Policy
    • Terms of Use
    • Data Policy

    Company

    • ConductScience
    • ConductScience Fellows
    © 2026 ConductScience, Inc. Research use only.