Executive Function
Automation Aid Task
Also called: automation bias, commission errors, omission errors, automated decision aid
Measure errors after false alerts and missed problems when an automated aid stays silent. Participants check readings with and without a programmed aid that is 80% correct.
5-8 minTimed: desktop or laptop with a mouse or trackpad
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 378
- Evidence grade
- C
- Registered clinical studies
- 2
What it measures
Measures errors made with an imperfect automated aid. Participants check three readings with and without programmed alerts. Scores wrong actions after false alerts and missed actions when the aid stays silent.
Constructs
- Automation bias
- Decision making
- Monitoring
Research uses
- Human factors research
- Reliance on automated advice
- Within-study comparisons
How the paradigm works
Automation bias is reliance on an automated aid when the available readings support another action. This task separates actions after false alerts from missed actions when an alert is absent.
More about the paradigm
Participants inspect three readings and choose Adjust if any is outside 40 to 60 inclusive, otherwise Leave. Eight practice checks give feedback. The standard session has two pairs of 20-check parts, one aided and one unaided in each pair, for 80 scored checks. Each aided part has eight correct alerts, eight correct silences, two false alerts and two missed alerts. The programmed aid is exactly 80% correct, with its errors set before the session. Each check allows 4,000 ms, configurable from 2,000 to 8,000 ms.
Scores
- Errors When the Aid Fails%
- Share of failed-aid checks with a false-alert action or a missed action after a silent aid. Timeouts count as missed actions, never as false-alert actions.
- Lower is fewer errors
- Actions After False Alerts%
- Adjust responses when all readings were in range but the aid alerted.
- Lower is fewer errors
- Misses Without an Alert%
- Leave responses or timeouts when a reading was out of range but the aid stayed silent.
- Lower is fewer misses
- Matched Errors Without the Aid%
- The same error measure on matching conditions without displayed advice.
- Lower is fewer errors
- Extra Errors With the Aid%
- Failed-aid error rate minus the matched unaided error rate.
- Positive means more errors with the aid
- Accuracy With the Aid%
- Correct choices across all aided checks, counting timeouts as incorrect.
- Higher is more accurate
- Accuracy Without the Aid%
- Correct choices across all unaided checks, counting timeouts as incorrect.
- Higher is more accurate
- Median Decision Timems
- Median response time after excluding anticipations.
- Informational
- Anticipationscount
- Responses under 100 ms, left out of rates.
- 0 expected
- Timeoutscount
- Checks with no response before the deadline.
- Informational
- Checks Completedtrials
- Scored checks presented, including timeouts.
- Informational
- Planned Checkstrials
- Checks planned for this run.
- Informational
Use and evidence
378 papers from 2021 to 2025 name this task in their title or abstract, up 440% compared with 2016 to 2020. That ranks it 74 of 582 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 212 |
| 2024 | 74 |
| 2023 | 53 |
| 2022 | 23 |
| 2021 | 16 |
| 2020 | 15 |
| 2019 | 20 |
| 2018 | 15 |
| 2017 | 13 |
| 2016 | 7 |
| 2015 | 13 |
| 2014 | 8 |
| 2013 | 1 |
| 2012 | 4 |
| 2011 | 10 |
| 2010 | 6 |
| 2009 | 2 |
| 2008 | 7 |
| 2007 | 3 |
| 2006 | 7 |
| 2005 | 6 |
| 2004 | 1 |
| 2003 | 4 |
| 2002 | 3 |
| 2001 | 2 |
| 2000 | 2 |
Use grade B
300 to 999 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 2
- Papers on reliability or validity
- 385
- Registered clinical studies using it as an outcome
- 2
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes clinical and AI-ethics commentary that runs no task.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Checks
- 80 scored checks as standard: 2 pairs of 20-check aided and unaided parts. Choose 1 to 3 pairs (40 to 120 checks). Eight practice checks with feedback come first.
- Rule
- Adjust if any of three readings is below 40 or above 60; Leave otherwise. Readings are integers from 31 to 69.
- Aid
- Programmed, with accuracy fixed in advance: 80% in each aided part. Of 20 checks, 8 show correct alerts, 8 correct silences, 2 false alerts and 2 missed alerts. Unaided parts have matching condition counts.
- Timing
- 4,000 ms per check as standard, from 2,000 to 8,000 ms. Practice feedback lasts 1,200 ms; scored choices have a 250 ms pause without feedback.
- Order
- Seeded check order and alternating order of aided and unaided parts within successive pairs. Every reading and alert state is saved.
- Scores
- Wrong actions after false alerts and missed actions without an alert, combined as the failed-aid error rate. Also matched unaided errors, their difference, overall accuracy, response time, timeouts and anticipations under 100 ms.
This version uses original numeric checks with a fixed rule and matched aided and unaided parts, instead of the flight simulation in Skitka and colleagues. The aid is a program with a preset error schedule. Scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored checks from the most complete run. Errors after wrong or absent alerts are the measure, so accuracy and repeated answers only flag for review. Anticipations under 100 ms are counted and excluded from rates. Incomplete runs are flagged; fewer than half the checks is invalid. Missed responses and responses under 100 ms follow the timed-test rule.
Why these rules
Following an incorrect aid is the behavior being measured. Low accuracy can therefore be a real response to the aid and must not exclude that behavior. A response before 100 ms cannot reflect inspection of three readings.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Pairs of parts | 2 | 1 to 3 |
| Time per check (ms) | 4,000 | 2,000 to 8,000 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Timed test: participants need a desktop or laptop with a mouse or trackpad. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Automation Aid Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Skitka LJ, Mosier KL, Burdick M, 1999), with the standard settings (pairs of parts: 2; time per check (ms): 4,000). The main score was errors when the aid fails (share of failed-aid checks with a false-alert action or a missed action after a silent aid).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Skitka, L. J., Mosier, K. L., & Burdick, M. (1999). Does automation bias decision-making? International Journal of Human-Computer Studies, 51(5), 991-1006. https://doi.org/10.1006/ijhc.1999.0252
Source papers
- Skitka, L. J., Mosier, K. L., & Burdick, M. (1999). Does automation bias decision-making? International Journal of Human-Computer Studies, 51(5), 991-1006.Paradigm source, Adults
Questions
How long does the Automation Aid Task take?
About 5 to 8 minutes, as listed in the test library.
What device does a participant need?
Timed test: participants need a desktop or laptop with a mouse or trackpad. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Errors When the Aid Fails, Actions After False Alerts, Misses Without an Alert, Matched Errors Without the Aid, Extra Errors With the Aid, Accuracy With the Aid, Accuracy Without the Aid, Median Decision Time, Anticipations, Timeouts, Checks Completed and Planned Checks. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Automation Aid Task used in research?
378 papers from 2021 to 2025 name it in their title or abstract, up 440% compared with 2016 to 2020. That ranks it 74 of 582 test families in the ConductCognition test census (use grade B). Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Automation Aid Task have?
In the published research, 2 papers report its test-retest reliability and 2 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Pairs of parts (1 to 3) and time per check (ms) (2,000 to 8,000). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.
Related tests