Social Cognition
Implicit Relational Assessment Task
Also called: implicit relational assessment procedure, IRAP, D-IRAP
Measure how readily people confirm or deny a relation: participants answer similar or opposite to a word and a picture under a rule that switches every block, scored by the D-IRAP.
8-12 minTimed: desktop or laptop with a mouse or trackpad
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 89
- Evidence grade
- C
- Registered clinical studies
- 1
X
XWhat it measures
Measures how readily people confirm or deny a relation from how fast they answer when a rule asks for the usual relation and when it asks for the reverse. A word (pleasant or unpleasant) and a picture of a fruit or an insect appear with the answers similar and opposite, and the rule switches every block.
Constructs
- Implicit relational responding
- Attitudes
- Speeded relational judgment
Research uses
- Social cognition and attitude research at the group level
- Relational frame theory and contextual behavioral science research
- Method studies of implicit measures
How the paradigm works
The Implicit Relational Assessment Task measures how readily people confirm or deny a relation between two things from how fast they answer when a rule asks for the usual relation and when it asks for the reverse. It follows the Implicit Relational Assessment Procedure (IRAP) as Barnes-Holmes, Barnes-Holmes, Stewart and Boles (2010) describe it. Each trial shows a label at the top of the screen, Pleasant or Unpleasant, a picture of a fruit or an insect in the middle, and the answers Similar and Opposite at the bottom left and right, whose sides change from trial to trial and never stay on one side more than twice in a row. D answers on the left and K on the right. A wrong key shows a red X until the right key is pressed, and each latency runs from the trial's onset to the right key. After the right key the screen clears for 400 ms.
More about the paradigm
In consistent blocks the right answer follows the usual relation (a fruit is similar to pleasant, an insect is opposite to it), and in inconsistent blocks the reverse; each block's rule is stated before it starts. Each block of 24 trials shows each of 12 Noto Emoji pictures, six fruits and six insects, once with each label, with no trial type twice in a row. Practice pairs (a block of each kind, which kind first drawn from the session's seed) repeat until both blocks of a pair have at least 80% of first answers right and a median latency under 2,000 ms, up to four pairs; a participant who does not reach the criteria ends the task there, as in the source. Six test blocks follow, alternating the two rules. A trial that reaches the latency criterion with no right answer shows Too slow. The criteria are settings: 2,000 to 3,000 ms, and 80% or 70%.
The main score is the overall D-IRAP: for each pair of test blocks the inconsistent minus the consistent mean latency, divided by the standard deviation of both blocks' latencies, averaged over the three pairs, after dropping latencies over 10,000 ms. The same steps within each of the four trial types (pleasant with fruit, pleasant with insects, unpleasant with fruit and unpleasant with insects) give the four trial-type D-IRAP scores. Positive scores mean faster answers under the consistent rule. Sessions are flagged when a test block falls below the accuracy criterion, a trial type's mean latency passes the latency criterion, or more than 10% of test latencies are under 300 ms; the source sets such participants aside. Answers whose first key comes under 100 ms are counted as anticipations. A D-IRAP is a research measure for group comparisons and says little about one person. Scores support research and do not diagnose.
Scores
- D-IRAPscore
- Inconsistent minus consistent mean latency over the standard deviation of both, for each pair of test blocks, averaged over the three pairs; latencies over 10,000 ms dropped.
- Positive values mean faster answers under the consistent rule (fruit pleasant, insects unpleasant); not better or worse
- D-IRAP, Pleasant and Fruitscore
- D-IRAP for trials with the label Pleasant and a fruit; the consistent answer is Similar.
- Positive values mean faster confirming that fruit is like pleasant
- D-IRAP, Pleasant and Insectsscore
- D-IRAP for trials with the label Pleasant and an insect; the consistent answer is Opposite.
- Positive values mean faster denying that insects are like pleasant
- D-IRAP, Unpleasant and Fruitscore
- D-IRAP for trials with the label Unpleasant and a fruit; the consistent answer is Opposite.
- Positive values mean faster denying that fruit is like unpleasant
- D-IRAP, Unpleasant and Insectsscore
- D-IRAP for trials with the label Unpleasant and an insect; the consistent answer is Similar.
- Positive values mean faster confirming that insects are like unpleasant
- Consistent Latencyms
- Mean latency to the right key in the consistent test blocks, up to 10,000 ms.
- Lower is faster
- Inconsistent Latencyms
- Mean latency to the right key in the inconsistent test blocks, up to 10,000 ms.
- Lower is faster
- Consistent Accuracy%
- Share of consistent test trials whose first key was right.
- Higher is better
- Inconsistent Accuracy%
- Share of inconsistent test trials whose first key was right.
- Higher is better
- Median Latencyms
- Median latency to the right key over all test trials.
- Informational
- Practice Pairscount
- Practice pairs run before the criteria were met or the four allowed were used.
- Informational
- Practice Criteria Met0/1
- 1 when both blocks of a practice pair met the criteria; 0 when no pair did and the task ended.
- 0 is flagged for review
- Test Blocks Under Accuracy Criterioncount
- Test blocks whose share of right first answers fell below the practice accuracy criterion.
- Above 0 is flagged for review
- Trial Types Over Latency Criterioncount
- Of the 8 mean latencies (four trial types under each rule), those over the latency criterion.
- Above 0 is flagged for review
- Too Slow Trialscount
- Test trials that reached the latency criterion with no right answer.
- Informational
- Fast Responses%
- Share of test latencies under 300 ms.
- Above 10% is flagged for review
- Anticipationscount
- Test trials whose first key came under 100 ms after the trial appeared.
- Quality check
- Slow Trials Droppedcount
- Test latencies over 10,000 ms, left out of the D-IRAP scores.
- Quality check
- Block Ordercount
- 1 when each pair starts with a consistent block, 2 when it starts with an inconsistent block, drawn from the session's seed.
- Informational
- Planned Test Trialstrials
- Test trials in six blocks of 24.
- Informational
- Total Test Trialstrials
- Test trials in the scored run.
- Fewer than planned is flagged for review
Use and evidence
89 papers from 2021 to 2025 name this task in their title or abstract, down 36% compared with 2016 to 2020. That ranks it 197 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 19 |
| 2024 | 13 |
| 2023 | 17 |
| 2022 | 20 |
| 2021 | 20 |
| 2020 | 35 |
| 2019 | 28 |
| 2018 | 19 |
| 2017 | 32 |
| 2016 | 25 |
| 2015 | 40 |
| 2014 | 17 |
| 2013 | 12 |
| 2012 | 16 |
| 2011 | 11 |
| 2010 | 18 |
| 2009 | 13 |
| 2008 | 5 |
| 2007 | 2 |
| 2006 | 2 |
| 2005 | 0 |
| 2004 | 0 |
| 2003 | 0 |
| 2002 | 0 |
| 2001 | 0 |
| 2000 | 0 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 5
- Papers on reliability or validity
- 69
- Registered clinical studies using it as an outcome
- 1
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Trials
- A label at the top (Pleasant or Unpleasant), a picture of a fruit or an insect, and the answers Similar and Opposite at the bottom left (D) and right (K). The answers change sides, never more than 2 trials in a row on one side.
- Pictures
- Noto Emoji pictures, 6 fruits and 6 insects. Only the labels and the answers are words, written in each app language.
- Blocks
- 24 trials each: every picture once with each label, with no trial type and no picture twice in a row. Consistent blocks treat fruit as pleasant and insects as unpleasant, inconsistent blocks the reverse; they alternate, and which comes first is drawn from the seed. Each block's rule is shown before it starts.
- Feedback
- A wrong key shows a red X until the right key. Too slow shows when 2,000 ms (standard; 2,000 to 3,000) pass with no right answer. The screen clears for 400 ms after the right key.
- Practice
- Pairs of blocks, repeated until both blocks of a pair have at least 80% (standard; or 70%) of first answers right and a median latency under the latency criterion, up to 4 pairs. A participant who does not get there ends the task.
- Test
- 6 blocks, 3 under each rule. Each block's accuracy and median latency show after it.
- Scoring
- D-IRAP for each pair of test blocks, after dropping latencies over 10,000 ms, averaged over the 3 pairs: overall and for each of the 4 trial types.
Barnes-Holmes and colleagues (2010) let participants learn the first block's rule from the red X and tell them only that the answers reverse. Here each block's rule is stated before it starts, because the task runs without an experimenter, and the targets are pictures, not words.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: every relational_responding_trial of the scored run, D or K for the answer that fits the block's rule (chance 50%; the right answer is on each side on half of every block's trials), correctness from the first key pressed. Below chance invalidates and not above chance is flagged. One key on 90% of first answers invalidates when accuracy is not above chance. Practice criteria not met, a test block below the accuracy criterion, a trial type's mean latency over the latency criterion, more than 10% of test latencies under 300 ms, or a short run is flagged for review. There is no response window, so nothing is missed.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). Barnes-Holmes and colleagues (2010) set aside participants who do not reach the practice criteria, who fall below the accuracy criterion in a test block or over the latency criterion in a trial type, or who give more than 10% of test latencies under 300 ms, because their latencies may not reflect following the rules quickly.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Latency criterion (ms) | 2,000 | 2,000 to 3,000 |
| Practice accuracy criterion | 80% (standard) | 80% (standard), 70% |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Timed test: participants need a desktop or laptop with a mouse or trackpad. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Implicit Relational Assessment Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Barnes-Holmes, Barnes-Holmes, Stewart & Boles, 2010), with the standard settings (latency criterion (ms): 2,000; practice accuracy criterion: 80%). The main score was D-IRAP (inconsistent minus consistent mean latency over the standard deviation of both, for each pair of test blocks, averaged over the three pairs; latencies over 10,000 ms dropped).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010). A sketch of the Implicit Relational Assessment Procedure (IRAP) and the Relational Elaboration and Coherence (REC) model. The Psychological Record, 60(3), 527-542. https://doi.org/10.1007/BF03395726
Source papers
- Barnes-Holmes D, Barnes-Holmes Y, Stewart I, Boles S (2010). A sketch of the Implicit Relational Assessment Procedure (IRAP) and the Relational Elaboration and Coherence (REC) model. The Psychological Record, 60(3):527-542.Paradigm source, Adults
- No normative data for this task.Paradigm source, N/A
Questions
How long does the Implicit Relational Assessment Task take?
About 8 to 12 minutes, as listed in the test library.
What device does a participant need?
Timed test: participants need a desktop or laptop with a mouse or trackpad. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
D-IRAP, D-IRAP, Pleasant and Fruit, D-IRAP, Pleasant and Insects, D-IRAP, Unpleasant and Fruit, D-IRAP, Unpleasant and Insects, Consistent Latency, Inconsistent Latency, Consistent Accuracy, Inconsistent Accuracy, Median Latency, Practice Pairs, Practice Criteria Met, Test Blocks Under Accuracy Criterion, Trial Types Over Latency Criterion, Too Slow Trials, Fast Responses, Anticipations, Slow Trials Dropped, Block Order, Planned Test Trials and Total Test Trials. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Implicit Relational Assessment Task used in research?
89 papers from 2021 to 2025 name it in their title or abstract, down 36% compared with 2016 to 2020. That ranks it 197 of 395 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Implicit Relational Assessment Task have?
In the published research, 5 papers report its test-retest reliability and 1 registered clinical study uses it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Latency criterion (ms) (2,000 to 3,000) and practice accuracy criterion (80% (standard), 70%). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.