Decision Making
Forecasting Task
Also called: judgmental forecasting task, time series forecasting task, graphical extrapolation task, trend extrapolation task
Measure judgmental forecasting: participants click the next points of charted series with a set trend, seasonal wave and noise, scored as the distance from the generating rule, trend damping and noise chasing.
6-10 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 87
- Evidence grade
- C
- Registered clinical studies
- 0
What it measures
Measures judgmental forecasting from graphs. Each chart shows a series generated with a set trend, seasonal wave and noise, and the participant clicks the values of the next points. The main score is the mean distance of the forecasts from the series' generating rule; trend damping and noise chasing are scored too.
Constructs
- Judgmental forecasting
- Trend extrapolation
- Sensitivity to noise
- Intuitive statistics
Research uses
- Judgment and decision-making research
- Forecasting and operations research
- Graph literacy and numeracy research
- Within-study comparison of forecasting conditions
How the paradigm works
The forecasting task measures judgmental forecasting: how people continue a series of numbers by eye (Harvey and Bolger, 1996; Lawrence, Goodwin, O'Connor and Önkal, 2006). Each chart shows 30 values of a series generated by our code, with 1 to 6 shaded columns to its right, and the participant clicks or taps the chart at the height expected for each next point, as in the web task of Harvey and Reimers (2013). Each series is a generating rule, a level with a straight trend and a seasonal wave, plus Gaussian noise. The 16 charts cross four structures (flat, rising, falling and seasonal) with two noise levels (a spread of 2 or 6 units on a 0 to 100 axis) and two variants: a steep and a shallow slope for the rising and falling series, and another level or phase for the others. Two practice charts with little noise show what came next; the test charts do not. There is no time limit, and the study sets how many points each chart asks for (4 as standard).
More about the paradigm
The main score is the forecast error: the mean distance of the forecasts from the generating rule, in units of the axis. Three known errors of judgmental forecasts are scored too. Trend damping: forecasts that rise less than a rising series and fall less than a falling one, scored as 1 minus the slope of the forecasts across the steps divided by the series' slope. Noise chasing: forecasts that stay with the last shown point instead of the rule, scored as the slope of the first forecast's error on the last point's distance from the rule. And how far forecasts follow the seasonal wave. In the review of Lawrence and colleagues (2006), people damp both up and down trends, down trends more, and treat randomness in a series as if it were signal. Harvey and Reimers (2013), with 20 series of 50 points online, found damping for steep trends and its opposite for shallow ones, both stronger with more noise. Scores describe forecasts of generated series; they are research measures and do not judge a person's forecasting at work.
Scores
- Forecast Error%
- Mean distance of the forecasts from the generating rule, in units of the 0 to 100 axis (percent of its height), over every chart and step.
- Lower means forecasts closer to the rule; the noise of each series sets no floor, since the rule itself is the target
- Trend Dampingscore
- On the rising and falling charts, 1 minus the mean of the forecasts' slope across the steps divided by the series' slope. Needs 2 or more points per chart.
- 0 follows the trend; 1 is flat forecasts; negative is steeper than the trend
- Noise Chasingscore
- Slope of the first forecast's error on the last shown point's distance from the rule, across charts. Within each structure and noise level, one chart's last point lies above the rule and the other's below it.
- 0 ignores where the last point fell; 1 moves with it fully
- Seasonal Wave Followedscore
- On the seasonal charts, the slope of the forecasts on the seasonal wave across the steps, within each chart. Needs 2 or more points per chart.
- 1 follows the wave; 0 ignores it
- Forecast Bias%
- Mean signed distance of the forecasts from the rule, in axis units.
- Positive means forecasts above the rule; near 0 is unbiased
- Forecast Error, Little Noise%
- Forecast error on the charts with a noise spread of 2 units.
- Lower is closer to the rule
- Forecast Error, More Noise%
- Forecast error on the charts with a noise spread of 6 units.
- Lower is closer to the rule; usually higher than with little noise
- Median Chart Timems
- Median time from a chart appearing to Done.
- Informational; the task has no time limit
- Anticipationscount
- Placements within 100 ms of a chart appearing. They are not taken.
- Quality check
- Charts Forecastcount
- Test charts with every point placed and confirmed in the scored run.
- Fewer than half the planned charts invalidates
- Planned Chartscount
- Test charts in a session: 16.
- Informational
Use and evidence
87 papers from 2021 to 2025 name this task in their title or abstract, down 16% compared with 2016 to 2020. That ranks it 299 of 670 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 24 |
| 2024 | 21 |
| 2023 | 13 |
| 2022 | 14 |
| 2021 | 15 |
| 2020 | 22 |
| 2019 | 23 |
| 2018 | 28 |
| 2017 | 15 |
| 2016 | 16 |
| 2015 | 25 |
| 2014 | 19 |
| 2013 | 19 |
| 2012 | 15 |
| 2011 | 10 |
| 2010 | 19 |
| 2009 | 7 |
| 2008 | 16 |
| 2007 | 11 |
| 2006 | 11 |
| 2005 | 17 |
| 2004 | 11 |
| 2003 | 5 |
| 2002 | 7 |
| 2001 | 7 |
| 2000 | 10 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 32
- Registered clinical studies using it as an outcome
- 0
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes organizational forecasting studies.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.48.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Charts
- 16 charts a session, in an order drawn from the session's seed. Each shows 30 values on a 0 to 100 axis and 1 to 6 shaded columns for the forecasts (4 as standard).
- Series
- Each series is a rule plus Gaussian noise: a level with a straight trend of 1.6 or 0.6 units a step, up or down, or a seasonal wave of 10 units every 6 steps. Four structures, flat, rising, falling and seasonal, cross two noise spreads, 2 and 6 units, and two variants.
- Last point
- Within each structure and noise level, one chart's last shown point lies above the rule and the other's below it, so following the last point cannot pass for a general lean.
- Forecasts
- A click or tap places the next point at its height; a drag, or a click in a shaded column, moves a point, and the arrow keys work too. Done confirms once every point is placed, with no time limit. Placements under 100 ms are not taken.
- Practice
- 2 charts with little noise, a steep rise and a seasonal wave, after which what came next appears as dark squares. The 16 test charts show nothing after Done.
- Scores
- Mean distance of the forecasts from the rule (the main score), trend damping, noise chasing, how far forecasts follow the seasonal wave, their bias, the error at each noise level and chart times.
The series and scores are this library's own, after the web task of Harvey and Reimers (2013), so their values do not carry over to other series or axes. The scores describe forecasts of generated series, support research and do not judge a person's forecasting at work.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: forecasting_chart from the run with the most charts, latest on ties, one per chart. A forecast's place has no correct option to compare with chance, so accuracy is not judged against chance. Forecasting fewer than half the planned charts invalidates; a run short of its planned charts is flagged. Charts confirmed within 200 ms of appearing on at least half the charts invalidate. A hidden tab is flagged.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast. Every measure compares forecasts across charts of different trends, noise and seasons, so at least half the charts are needed to read them. Large errors, damped trends and forecasts that follow the last point are the results of interest and never invalidate.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Points to forecast | 4 | 1 to 6 |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops and tablets. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Forecasting Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Harvey & Bolger, 1996; Harvey & Reimers, 2013), with the standard settings (points to forecast: 4). The main score was forecast error (mean distance of the forecasts from the generating rule, in units of the 0 to 100 axis (percent of its height), over every chart and step).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Harvey, N., & Bolger, F. (1996). Graphs versus tables: Effects of data presentation format on judgemental forecasting. International Journal of Forecasting, 12(1), 119-137. https://doi.org/10.1016/0169-2070(95)00634-6Judgmental forecasts from series shown as graphs or as tables; this task shows graphs.
- Lawrence, M., Goodwin, P., O'Connor, M., & Önkal, D. (2006). Judgmental forecasting: A review of progress over the last 25 years. International Journal of Forecasting, 22(3), 493-518. https://doi.org/10.1016/j.ijforecast.2006.03.007Review of how judgmental forecasts respond to trend, seasonality and noise, damping trends and treating noise as signal.
- Harvey, N., & Reimers, S. (2013). Trend damping: Under-adjustment, experimental artifact, or adaptation to features of the natural environment? Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(2), 589-607. https://doi.org/10.1037/a0029179The web procedure, clicks in the columns after a series at two noise levels, and damping for steep trends with its opposite for shallow ones.
- Reimers, S., & Harvey, N. (2011). Sensitivity to autocorrelation in judgmental time series forecasting. International Journal of Forecasting, 27(4), 1196-1214. https://doi.org/10.1016/j.ijforecast.2010.08.004Forecasts that treat independent noise as if it carried over, the basis of the noise chasing score.
Source papers
- Harvey N, Reimers S (2013). Trend damping: Under-adjustment, experimental artifact, or adaptation to features of the natural environment? Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(2):589-607.N 662, Adults (online)
- No normative data for this task.Paradigm source, N/A
Questions
How long does the Forecasting Task take?
About 6 to 10 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops and tablets. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Forecast Error, Trend Damping, Noise Chasing, Seasonal Wave Followed, Forecast Bias, Forecast Error, Little Noise, Forecast Error, More Noise, Median Chart Time, Anticipations, Charts Forecast and Planned Charts. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Forecasting Task used in research?
87 papers from 2021 to 2025 name it in their title or abstract, down 16% compared with 2016 to 2020. That ranks it 299 of 670 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.48.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Forecasting Task have?
In the published research, 0 papers report its test-retest reliability and 0 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Points to forecast (1 to 6). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.