Fixed order
Steps run in the order shown.
Study design
Randomize test order and assign study groups
Put tests in sections, choose the order inside each section, and assign participants to study groups. Each participant's group and order are fixed when they agree to the consent form, and both go into your exports.
In the protocol builder
- Sections
- Up to 20, holding tests, instructions and breaks
- Order inside a section
- Fixed order, Random order or Latin square
- Section order
- Fixed, or rotated in a Latin square
- Study groups
- Up to 12. Random, weighted or balanced assignment, with optional caps
- Saved for each participant
- Group, order number and the tests in the order received
See each participant's path
The protocol builder's Flow view draws the path from consent to the end for each study group, with a text version below it. This is the example study used on this page, drawn by the same component. Practice stays first, the attention tests rotate in a Latin square, and each group gets one more section.
Text version of the diagram
- Consent: Participant agrees to take part.
- Balanced assignment to Group A, Group B.
- Group A: Practice (1 test, fixed order); Attention (3 tests, Latin square); Speed (1 test, fixed order).
- Group B: Practice (1 test, fixed order); Attention (3 tests, Latin square); Memory and speed (2 tests, random order).
- Group A: Attention, Speed rotate across participants in a Latin square with 2 orders.
- Group B: Attention, Memory and speed rotate across participants in a Latin square with 2 orders.
- End: completion page.
Choose the order inside each section
A section is a named block of tests, instructions and breaks. Each section has one order rule.
Random order
Each participant gets the tests in a new random order. Instructions and breaks stay where they are.
Latin square
Tests rotate across participants in a balanced Latin square, so each test is used in each position equally often. A section needs 2 to 12 tests. Instructions and breaks stay where they are.
Instructions are plain text of up to 5,000 characters. A break can ask for up to 900 seconds of rest (15 minutes), with or without an early Continue. A protocol can hold up to 20 sections and 128 steps.
Counterbalance whole sections
Section order can rotate in a balanced Latin square too. Keep a section in place, such as practice at the start, and the other sections rotate around it.
When a study counterbalances more than one thing, each participant gets one order number and every part stays balanced. In the example study, the 3 attention tests have 6 orders and the 2 sections after practice have 2, so each group rotates through 6 orders. A group can have up to 1,000 orders.
Latin square counterbalancing, explained
When every participant takes the tests in the same order, practice and fatigue build up in the same tests for everyone. Counterbalancing spreads those order effects across the tests, so they do not pile up on one.
A Latin square is a set of orders in which each test appears once in each position. A balanced Latin square, also called a Williams design, adds one more rule: each test comes directly after every other test equally often. ConductCognition builds balanced Latin squares.
With an even number of tests, a balanced square needs as many orders as tests. With an odd number, it needs twice as many. Each new participant gets the order that the fewest active participants in their group have, so the orders fill evenly. For an exactly balanced design, plan a number of participants per group that is a multiple of the number of orders.
Random order is the other choice. It gives each participant a new shuffle, so positions even out only on average across many participants.
Williams, E. J. (1949). Experimental designs balanced for the estimation of residual effects of treatments. Australian Journal of Scientific Research, Series A, 2(2), 149-168. https://doi.org/10.1071/CH9490149
| Order | 1st | 2nd | 3rd | 4th |
|---|---|---|---|---|
| 1 | A | B | D | C |
| 2 | B | C | A | D |
| 3 | C | D | B | A |
| 4 | D | A | C | B |
Orders a balanced square needs
- 2 tests: 2 orders
- 3 tests: 6 orders
- 4 tests: 4 orders
- 5 tests: 10 orders
- 6 tests: 6 orders
- 7 tests: 14 orders
- 8 tests: 8 orders
Assign participants to study groups
Study groups, which clinical teams call arms, each receive their own sections. A participant joins one group when they agree to the consent form and keeps it, even when you publish a new protocol version.
| Assignment | How it works |
|---|---|
| Random | Each participant has an equal chance of joining each group. |
| Weighted | Each group's chance follows its ratio. |
| Balanced | Each participant joins the group furthest below its ratio. Someone who has not started a test 60 minutes after consent stops counting, so their place is reused. |
| Caps | A cap limits a group's active participants. A full group takes no one new. Someone who has not started a test 60 minutes after consent frees their place. |
A study can have up to 12 groups, each with a ratio from 1 to 100 and an optional cap of up to 100,000 participants. Groups differ by the sections they receive. To give two groups the same test with different settings, put each version in a section that only one group receives.
Before launch, Simulate assignments in the Groups view shows how a number of participants would spread across groups and orders. People join one after another and every one starts. Nothing is saved and no places or credits are used.
Running a trial with arms? See cognitive tests in clinical trials.
Preview as any group and order
Choose a study group and an order number, then run the whole protocol through the screens participants see. The preview opens in a new tab. It creates no participant and uses no places or credits.
What the exports record
Score and trial CSV exports add these columns for every participant, after the columns they already had.
| Column | What it holds |
|---|---|
| group_key | The study group's key. |
| group_name | The group's name in the protocol version the participant ran. |
| assignment_method | How groups were assigned: random, weighted or balanced. |
| order_index | The participant's order number, counted from 1 as in the app. |
| section_label | The section this test was in. |
| sequence_position | Where this test came among the participant's tests. |
| realized_sequence | The participant's tests in the order they received them, joined by >. |
Example: the first 8 participants
Simulated participants in the example study, placed by the same assignment code the app runs at consent. Example data, not real participants.
| Participant | group_name | assignment_method | order_index | realized_sequence |
|---|---|---|---|---|
| 1 | Group A | balanced | 1 | srt>flanker>stroop>gonogo>crt |
| 2 | Group B | balanced | 5 | srt>flanker>gonogo>stroop>digit-span>crt |
| 3 | Group B | balanced | 1 | srt>flanker>stroop>gonogo>crt>digit-span |
| 4 | Group A | balanced | 3 | srt>gonogo>flanker>stroop>crt |
| 5 | Group A | balanced | 4 | srt>crt>gonogo>stroop>flanker |
| 6 | Group B | balanced | 2 | srt>crt>digit-span>stroop>gonogo>flanker |
| 7 | Group A | balanced | 6 | srt>crt>stroop>flanker>gonogo |
| 8 | Group B | balanced | 6 | srt>digit-span>crt>stroop>flanker>gonogo |
Every column and file: research data export. The tests in the example: the full test list.
Not built: branching and scheduled visits
Branching and scheduled visits are not built. A score or an answer cannot send a participant down a different path, and the app does not schedule return visits on set days. To test the same people twice, run each visit as its own study with the same participant IDs, and join two studies with the two-study comparison export.
Questions about study design
What is the difference between random order and a Latin square?
Random order shuffles the tests for each participant, so each test lands in each position only on average. A Latin square rotates the tests through a fixed set of orders, so each test is used in each position equally often, and each new participant gets the order the fewest active participants in their group have.
How many orders does a balanced Latin square need?
As many orders as tests when the number of tests is even, and twice as many when it is odd. 4 tests need 4 orders; 5 tests need 10.
When is a participant assigned to a group and an order?
When they agree to the consent form. Their group, order number and exact sequence of tests are saved then and do not change, even when you publish a new protocol version.
What happens when a group is full?
A full group takes no one new. When every group has a cap and all are full, the open study link turns new people away before they start, and a participant link shows that the study is full before consent. Someone who has not started a test 60 minutes after consent stops counting toward caps and balance.
Can a score decide what a participant does next?
No. Branching on a score or an answer is not built. Every participant in a group gets that group's sections.
Can participants come back for a second visit?
Scheduled visits are not built. Run each visit as its own study with the same participant IDs, and use the two-study comparison export to join two of them by participant and test.
Do sections and study groups cost extra?
No. They work on every plan, including the free plan. A participant counts once toward your participant limit, or uses one credit, whatever their group or order.
Plans and prices: pricing.
Design your study
Start free, build a protocol with sections and groups, and preview it as each group.