Attention
Temporal Order Judgment Task
Also called: TOJ, temporal order judgement, simultaneity judgment, visual temporal order task
Measure the just noticeable difference and the point of subjective simultaneity for two brief flashes, with asynchronies set in whole screen frames.
7-9 minTimed: desktop or laptop with a mouse or trackpad
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Preview any test free after you sign up.
- Papers, 2021 to 2025
- 251
- Evidence grade
- C
- Registered clinical studies
- 4
What it measures
Measures how far apart in time two brief flashes must be for a person to tell which came first. Two disks flash left and right of a cross, 17 to 200 ms apart, and participants press an arrow key for the side that flashed first.
Constructs
- Temporal resolution of perception
- Time perception
- Visual attention
Research uses
- Perception and timing research
- Within-study group comparison
- Individual differences in temporal processing
How the paradigm works
The temporal order judgment task measures how far apart in time two events must be for a person to tell which came first. Two white disks flash briefly, one on each side of a fixation cross, at onset asynchronies of about 17 to 200 ms, with either disk first. Participants press the left or right arrow key for the side that flashed first. In the classic study of Hirsh and Sherrick (1961), trained observers needed about 20 ms to judge order.
More about the paradigm
The share of "right first" answers rises from near 0 to near 1 across the asynchronies. A cumulative Gaussian fitted by maximum likelihood gives the just noticeable difference, half the distance between its 25% and 75% points, and the point of subjective simultaneity, the asynchrony at which both answers are equally likely (Vroomen and Keetels, 2010). Shore, Spence and Klein (2001) used visual order judgments of this kind to show that attended stimuli are perceived earlier.
Asynchronies are planned in frames of a 60 Hz screen, shown as whole frames at each trial's measured refresh rate, and measured from the frames that drew the disks. A simultaneity version, set in the settings, asks whether the two flashes came at the same time and reports the center and width of the window of simultaneity.
Scores
- Just Noticeable Differencems
- Half the distance between the 25% and 75% points of the cumulative Gaussian fitted to the share of right-first answers.
- Lower means finer timing
- Point of Subjective Simultaneityms
- Asynchrony at which right-first and left-first answers are equally likely; positive when the right disk had to lead.
- Near 0 means no side bias
- Lapse Rate%
- Fitted share of answers that did not depend on the flashes.
- Lower means fewer lapses
- Accuracy%
- Share of order judgments that named the side that flashed first.
- Higher is better
- Accuracy by Asynchrony%
- Share correct at each asynchrony in ms, both sides together.
- Rises with the asynchrony
- Right First by Asynchrony%
- Share of right-first answers at each signed asynchrony in ms; negative asynchronies had the left disk first.
- The psychometric function
- Window Centerms
- Simultaneity version: asynchrony at which the fitted share of same-time answers peaks.
- Near 0 means no side bias
- Window Widthms
- Simultaneity version: full width at half maximum of the Gaussian window fitted to the same-time answers.
- Lower means a narrower window
- Window Peak%
- Simultaneity version: fitted share of same-time answers at the window center.
- Informational
- Same Time by Asynchrony%
- Simultaneity version: share of same-time answers at each signed asynchrony in ms.
- Highest near 0
- Fit R-squared%
- Share of the variance across asynchronies that the fitted curve explains, weighted by answers.
- Near 1 means a close fit
- Median RTms
- Median response time from the second flash, over responses at or after 100 ms.
- Lower is faster
- Dropped Frame Rate%
- Share of flash frames the browser dropped.
- Above 0.05 is flagged for review
Use and evidence
251 papers from 2021 to 2025 name this task in their title or abstract, down 24% compared with 2016 to 2020. That ranks it 100 of 395 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 56 |
| 2024 | 41 |
| 2023 | 53 |
| 2022 | 47 |
| 2021 | 54 |
| 2020 | 71 |
| 2019 | 63 |
| 2018 | 66 |
| 2017 | 73 |
| 2016 | 59 |
| 2015 | 72 |
| 2014 | 70 |
| 2013 | 71 |
| 2012 | 83 |
| 2011 | 55 |
| 2010 | 54 |
| 2009 | 35 |
| 2008 | 48 |
| 2007 | 37 |
| 2006 | 34 |
| 2005 | 28 |
| 2004 | 27 |
| 2003 | 17 |
| 2002 | 13 |
| 2001 | 7 |
| 2000 | 8 |
Use grade C
100 to 299 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 5
- Papers on reliability or validity
- 91
- Registered clinical studies using it as an outcome
- 4
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Display
- Two white disks 24 pixels wide, 110 pixels left and right of a gray cross on black.
- Flashes
- Each disk flashes for 2 frames of a 60 Hz screen (about 33 ms), shown as whole frames at the measured refresh rate.
- Asynchronies
- 1, 2, 3, 4, 6, 8 and 12 frames of a 60 Hz screen (about 17 to 200 ms), each with either side first, mixed in one random order.
- Fixation
- 600 to 1,000 ms, drawn at random for each trial.
- Answer
- From the second flash until 3,000 ms after it.
- Keys
- Left and right arrows for the side that flashed first; S and D for the same or different times in the simultaneity version.
- Length
- 12 trials per asynchrony (6 to 16), 168 trials with a break after every 56.
- Practice
- 8 trials with feedback at the two longest asynchronies; half of them simultaneous in the simultaneity version.
- Fit
- A cumulative Gaussian fitted by maximum likelihood, with a lapse rate of 0 to 0.06.
- Timing
- The shown asynchrony is measured from the frames that drew the disks; more than 5% dropped frames is flagged for review.
Hirsh and Sherrick (1961) compared order judgments across vision, hearing and touch. This version is visual only, since pairs with sounds need calibrated audio. On screens that are not 60 Hz the asynchronies are the nearest whole frames to the planned times, and the shown asynchrony is saved with every answer.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: temporal_order_test from the scored run, left or right arrow for the side that flashed first (chance 50%, each asynchrony equally often with each side first) or, in the simultaneity version, S or D for the same or different times (no chance level, since short asynchronies look simultaneous), from the second flash until 3,000 ms after it. Order judgments below chance invalidate and not above chance is flagged. One key on 90% of answers invalidates when accuracy is not above chance, and always in the simultaneity version. Missed responses and responses under 100 ms follow the timed-test rule. More than 5% of flash frames dropped, or no JND or no window from the fit, is flagged for review.
Why these rules
Accuracy significantly below chance (one-sided exact test, p < .05) cannot come from guessing; it means the response mapping was reversed or the task was not followed (Slick, Sherman and Iverson 1999 treat below-chance forced-choice performance as definite response bias). Chance-level accuracy can be a real result in impaired participants, so it is only flagged. One option on at least 90% of answered trials, when the correct options are balanced and accuracy is not above chance, means the answers did not depend on the items (the task counterpart of long-string responding in surveys; Curran 2016). Missed responses and responses under 100 ms keep the platform's timed-test rule: more than max(3, 25%) missed responses or more than max(2, 20%) responses under 100 ms invalidate, and any is flagged. A response under 100 ms cannot be a reaction to the stimulus. The asynchronies are whole screen frames, so a display that drops frames changes what was shown. In the simultaneity version the asynchronies run from 0 to 200 ms, so one answer on nearly every trial does not depend on the display.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Trials per asynchrony | 12 | 6 to 16 |
| Judgment | Which came first (standard) | Which came first (standard), Same time or different times |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Timed test: participants need a desktop or laptop with a mouse or trackpad. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Temporal Order Judgment Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Hirsh and Sherrick, 1961; Vroomen and Keetels, 2010), with the standard settings (trials per asynchrony: 12; judgment: Which came first). The main score was just noticeable difference (half the distance between the 25% and 75% points of the cumulative Gaussian fitted to the share of right-first answers).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Hirsh, I. J., & Sherrick, C. E., Jr. (1961). Perceived order in different sense modalities. Journal of Experimental Psychology, 62(5), 423-432. https://doi.org/10.1037/h0045283
- Shore, D. I., Spence, C., & Klein, R. M. (2001). Visual prior entry. Psychological Science, 12(3), 205-212. https://doi.org/10.1111/1467-9280.00337Visual order judgments between stimuli left and right of fixation.
- Vroomen, J., & Keetels, M. (2010). Perception of intersensory synchrony: A tutorial review. Attention, Perception, & Psychophysics, 72(4), 871-884. https://doi.org/10.3758/APP.72.4.871Definitions of the JND, the PSS and the simultaneity window.
Source papers
- Hirsh IJ, Sherrick CE (1961). Perceived order in different sense modalities. Journal of Experimental Psychology, 62(5):423-432.Paradigm source, Adults
- Shore DI, Spence C, Klein RM (2001). Visual prior entry. Psychological Science, 12(3):205-212.Paradigm source, Adults
- Vroomen J, Keetels M (2010). Perception of intersensory synchrony: A tutorial review. Attention, Perception, & Psychophysics, 72(4):871-884.Paradigm source, Adults
Questions
How long does the Temporal Order Judgment Task take?
About 7 to 9 minutes, as listed in the test library.
What device does a participant need?
Timed test: participants need a desktop or laptop with a mouse or trackpad. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Just Noticeable Difference, Point of Subjective Simultaneity, Lapse Rate, Accuracy, Accuracy by Asynchrony, Right First by Asynchrony, Window Center, Window Width, Window Peak, Same Time by Asynchrony, Fit R-squared, Median RT and Dropped Frame Rate. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Temporal Order Judgment Task used in research?
251 papers from 2021 to 2025 name it in their title or abstract, down 24% compared with 2016 to 2020. That ranks it 100 of 395 test families in the ConductCognition test census (use grade C). Source: ConductCognition test census 0.14.0, counted 31 October 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Temporal Order Judgment Task have?
In the published research, 5 papers report its test-retest reliability and 4 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Which settings can a researcher change?
Trials per asynchrony (6 to 16) and judgment (Which came first (standard), Same time or different times). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.