Social Cognition
Ironic Remarks Task
Also called: irony comprehension task, verbal irony task, ironic praise and criticism task
Measure understanding of ironic praise and criticism. Participants read short stories and choose what a friend means, with literal remarks for comparison.
4-6 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 156
- Evidence grade
- D
- Registered clinical studies
- 0
A friend sees tea spill across the table you have just cleaned.
Your friend says: “Very tidy.”
What does your friend mean?
The table is tidy.The table is messy.What it measures
Measures how well people understand ironic remarks in short stories. Participants choose what a speaker means. Ironic praise and criticism are compared with literal remarks. Scores describe understanding of these written stories.
Constructs
- Understanding verbal irony
- Pragmatic language
- Use of social context
Research uses
- Nonliteral language research
- Social cognition studies
- Aging research
- Within-study group comparison
How the paradigm works
The Ironic Remarks Task measures how well people use a story to understand an ironic remark. Participants read an everyday event followed by a friend's words, then choose what the friend means. The task includes ironic praise as well as ironic criticism, following the distinction studied by Bruntsch and Ruch (2017). Literal praise and criticism provide a comparison.
More about the paradigm
The standard session has 16 original stories, four in each condition. A setting uses 8 stories, two per condition. The seed selects and shuffles stories without repetition and balances correct answer positions within each condition. Two separate practice stories explain the intended meaning. There is no time limit and no correctness feedback during the test. All stories, remarks and choices follow the session language in eight languages.
The primary score is accuracy on answered ironic stories. Literal accuracy, separate praise and criticism scores, and the ironic-minus-literal difference show where errors occurred. Accuracy is based on two choices, with a 50% chance level. Median times include reading and choosing; compare them within one device type. The fixed story sets support comparisons on these items; they are not matched versions of the same event across conditions.
Scores
- Ironic Meaning Accuracy%
- Correct meanings among answered ironic stories.
- Higher means more ironic meanings understood
- Literal Meaning Accuracy%
- Correct meanings among answered literal stories.
- Higher supports interpretation of the irony score
- Ironic Minus Literal Accuracy%
- Ironic accuracy minus literal accuracy, in percentage points.
- Negative values mean lower accuracy on ironic stories
- Ironic Praise Accuracy%
- Correct meanings among answered ironic praise stories.
- Higher means more ironic praise understood
- Ironic Criticism Accuracy%
- Correct meanings among answered ironic criticism stories.
- Higher means more ironic criticism understood
- Overall Accuracy%
- Correct meanings among all answered stories.
- Higher means more meanings understood
- Median Ironic Response Timems
- Median time to read and answer an ironic story, excluding times under 100 ms.
- Compare within a device type
- Median Literal Response Timems
- Median time to read and answer a literal story, excluding times under 100 ms.
- Compare within a device type
- Stories Answeredcount
- Stories with a recognized answer and a saved condition.
- Completion
- Stories Setcount
- Stories planned for the session.
- Informational
- Answer Rate%
- Answered stories as a percent of those planned.
- Below 75% invalidates
- Anticipationscount
- Answers under 100 ms.
- Lower supports interpretation
Use and evidence
156 papers from 2021 to 2025 name this task in their title or abstract, up 46% compared with 2016 to 2020. That ranks it 184 of 582 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 40 |
| 2024 | 26 |
| 2023 | 27 |
| 2022 | 37 |
| 2021 | 26 |
| 2020 | 21 |
| 2019 | 21 |
| 2018 | 20 |
| 2017 | 25 |
| 2016 | 20 |
| 2015 | 10 |
| 2014 | 16 |
| 2013 | 24 |
| 2012 | 12 |
| 2011 | 8 |
| 2010 | 12 |
| 2009 | 6 |
| 2008 | 7 |
| 2007 | 8 |
| 2006 | 4 |
| 2005 | 2 |
| 2004 | 3 |
| 2003 | 3 |
| 2002 | 2 |
| 2001 | 0 |
| 2000 | 5 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade D
Little published measurement evidence found.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 19
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes some computational studies of sarcasm in text.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Stories
- 16 original stories: 4 each of ironic praise, ironic criticism, literal praise and literal criticism. A session shows each selected story once.
- Session
- 16 stories by default, or 8 as a setting. Each condition contributes equally. The seed selects and shuffles stories and balances the correct answer position within each condition.
- Answer
- The story and remark stay on screen while the participant chooses one of two meanings. There is no time limit; response time includes reading and choosing.
- Practice
- 2 separate stories, one ironic and one literal. Feedback explains the intended meaning. Test stories have no correctness feedback.
- Timing
- A 500 ms lead-in and a 300 ms gap after each choice. These are display pauses, not response deadlines.
- Languages
- Stories, remarks and choices are written in English, Italian, German, Spanish, French, Japanese, Portuguese and Chinese.
Scores describe understanding of these written stories, not a diagnosis or a general social skill. The two-choice format has 50% chance accuracy. Compare the same language and settings; no published items or norms are reused.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: ironic_remarks_choice from the most complete run. Fewer than 75% of planned stories answered invalidates; other incomplete runs and literal accuracy below 50% are flagged. Answers under 200 ms on at least half of 4 or more answers invalidate. A hidden tab is flagged. Low irony accuracy itself is not an exclusion.
Why these rules
The two choices have 50% chance accuracy. Low irony accuracy is the outcome measured; literal control accuracy is checked separately. Completion and literal-control thresholds are ConductScience choices. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Stories | 16 stories (standard) | 8 stories, 16 stories (standard) |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Ironic Remarks Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Bruntsch R, Ruch W, 2017), with the standard settings (stories: 16 stories). The main score was ironic meaning accuracy (correct meanings among answered ironic stories).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Bruntsch, R., & Ruch, W. (2017). Studying irony detection beyond ironic criticism: Let's include ironic praise. Frontiers in Psychology, 8, 606. https://doi.org/10.3389/fpsyg.2017.00606Rationale for including ironic praise and criticism alongside literal remarks. Our stories and direct two-choice format are original.
Source papers
- Bruntsch, R., & Ruch, W. (2017). Studying irony detection beyond ironic criticism: Let's include ironic praise. Frontiers in Psychology, 8, 606.Paradigm source, Adults in the source study
Questions
How long does the Ironic Remarks Task take?
About 4 to 6 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Ironic Meaning Accuracy, Literal Meaning Accuracy, Ironic Minus Literal Accuracy, Ironic Praise Accuracy, Ironic Criticism Accuracy, Overall Accuracy, Median Ironic Response Time, Median Literal Response Time, Stories Answered, Stories Set, Answer Rate and Anticipations. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Ironic Remarks Task used in research?
156 papers from 2021 to 2025 name it in their title or abstract, up 46% compared with 2016 to 2020. That ranks it 184 of 582 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.39.0, counted 8 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Ironic Remarks Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade D in the ConductCognition test census).
Which settings can a researcher change?
Stories (8 stories, 16 stories (standard)). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.