Reasoning
Free Sorting Task
Also called: free-sorting task, similarity sorting, pile sorting, open sort
Measure how people group things when no rule is given: participants sort 16 pictures or words into as many groups as they like, by drag, tap or keyboard.
3-6 minUntimed
Free to start: 10 free participants per account, then $0.99 per participant for academic research, or any yearly plan. Try it here first, with no account.
- Papers, 2021 to 2025
- 48
- Evidence grade
- C
- Registered clinical studies
- 3
What it measures
Measures how people group things when no rule is given. Participants put 16 everyday things into as many groups as they like, by dragging them, by tapping or with the keyboard. Scores the number of groups, how closely the groups follow the things' kinds, and which things were put together.
Constructs
- Categorization
- Conceptual organization
- Perceived similarity
- Taxonomic and thematic relations
Research uses
- Categorization and concept research
- Cultural and cross-language research
- Sensory and consumer research methods
- Within-study group comparison
How the paradigm works
The Free Sorting Task asks how a person groups things when no rule is given. All 16 things, everyday animals, foods, vehicles and clothes, appear at once in an area of things not sorted yet, in an order drawn for each session. The participant puts them into as many groups as they like, with as many things in each, by dragging each one into a group or onto New group, by tapping it and then where it goes, or with the keyboard. Things can move again until the participant chooses Done, which needs every thing in a group and at least 2 groups. There is no time limit, and a practice board of 4 things comes first.
More about the paradigm
The format is the free sorting task of sensory and categorization research (Lawless, Sheng and Knoops 1995; Chollet, Lelièvre, Abdi and Valentin 2011): the whole set at once, grouped by whatever the person sees in common. Chollet and colleagues advise 9 to 20 things and at least about 20 participants. Each person's sort is a matrix with a 1 for each pair put together and a 0 for each pair kept apart; summed over a study's participants, it is the similarity matrix that multidimensional scaling turns into a map, on which things often sorted together sit close. This task uses 16 things chosen for it, Noto Emoji pictures or, with the words setting, the things' everyday names in the participant's language, the same for every participant. Its four kinds cross with themes, such as a cow, an egg and a tractor on a farm, so a sort can follow kind or theme.
The main score is the number of groups. Kind agreement, the adjusted Rand index between the groups and the four kinds (Hubert and Arabie 1985), is 1 for a sort exactly by kind and about 0 for no more agreement than chance. The co-occurrence matrix holds each pair's 1 or 0 for the study's own map. Scores support research and do not diagnose.
Scores
- Groupscount
- Number of groups made.
- Informational; no number is better
- Kind Agreementscore
- Adjusted Rand index between the groups and the four kinds of the things (animals, foods, vehicles and clothes): 1 for a sort exactly by kind, about 0 for chance agreement.
- Informational; neither end is better
- Groups of Onecount
- Groups that hold a single thing.
- Informational
- Largest Groupitems
- Things in the largest group.
- Informational
- Things Sorteditems
- Things in a group, of 16.
- Informational
- Sorting Timems
- Time from the board appearing to Done.
- Informational
- Movesmoves
- Things moved, a thing moved twice counted twice.
- Informational
Use and evidence
48 papers from 2021 to 2025 name this task in their title or abstract, down 27% compared with 2016 to 2020. That ranks it 439 of 670 test families in our census. See the counts for every test.
Show the numbers
| Year | Papers |
|---|---|
| 2025 | 5 |
| 2024 | 7 |
| 2023 | 14 |
| 2022 | 11 |
| 2021 | 11 |
| 2020 | 16 |
| 2019 | 15 |
| 2018 | 13 |
| 2017 | 10 |
| 2016 | 12 |
| 2015 | 15 |
| 2014 | 15 |
| 2013 | 9 |
| 2012 | 11 |
| 2011 | 4 |
| 2010 | 7 |
| 2009 | 8 |
| 2008 | 2 |
| 2007 | 7 |
| 2006 | 8 |
| 2005 | 5 |
| 2004 | 4 |
| 2003 | 4 |
| 2002 | 3 |
| 2001 | 3 |
| 2000 | 2 |
Use grade D
30 to 99 papers from 2021 to 2025.
Evidence grade C
At least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies.
- Papers on test-retest reliability
- 0
- Papers on reliability or validity
- 14
- Registered clinical studies using it as an outcome
- 3
Measured on ConductCognition
Pooled across the studies that feed our platform norms, by device group. A figure needs 50 or more people.
No pooled figure yet.
Note on the counts: includes sensory and consumer studies of products.
How the grades work
Use: A is 1,000 or more papers from 2021 to 2025, B 300 to 999, C 100 to 299, D 30 to 99 and E fewer than 30. Evidence: A means at least 25 papers report its test-retest reliability and at least 25 registered clinical studies use it as an outcome; B, at least 8 test-retest papers, and either 5 registered clinical studies or 150 reliability or validity papers; C, at least 2 test-retest papers, 30 reliability or validity papers or 3 registered clinical studies; D, little published measurement evidence found. The counts measure use. They include every paper that names the task in its title or abstract. Grades describe the published research on this task, not ConductCognition's version of it.
Source: ConductCognition test census 0.48.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
Task design
The trial sequence as the test runs in ConductCognition.
- Things
- 16 everyday things, 4 animals, 4 foods, 4 vehicles and 4 clothes, as Noto Emoji pictures (standard), or their names in the participant's language as a setting, all on the board at once in an order drawn for each session.
- Sorting
- As many groups as the participant likes, with as many things in each. A thing moves by a drag, by a tap and then a tap where it goes, or with Tab and Enter, and can move again until Done.
- Done
- Works once every thing is in a group and there are at least 2 groups. No time limit.
- Themes
- The kinds cross with themes, such as a cow, an egg and a tractor on a farm, so a sort can follow kind or theme.
- Practice
- A board of 4 things, two fruits and two tools, before the test.
- Map
- Each person's 1 or 0 for each of the 120 pairs, summed over a study's participants, is the similarity matrix for multidimensional scaling.
The things are chosen for this test with Noto Emoji pictures and our own words in each language. No sort is right, and the number of groups describes how a person sorts, not how well. The scores support research and do not diagnose.
Quality checks
Each session is marked valid, needs review or invalid, with the reason, before you see its scores.
Scored rows: the free_sorting_board row of the scored run, one board with every thing's group and each move. A sort has no correct grouping, so accuracy is not judged against chance. A board with fewer than all 16 things sorted invalidates, and every thing in a group of its own is flagged. Last moves made under 200 ms after the move before, for at least half the things, invalidate.
Why these rules
There is no correct option to compare with chance, so accuracy is not judged against chance. A board with things left out has no group for them, so its pairs are missing. Sixteen groups of one put no two things together, which a participant may mean, so it is flagged rather than invalidated. An answer under 200 ms is faster than a simple visual reaction, so it cannot reflect the item. Following response-time effort screening for untimed tests (Wise and Kong 2005), the session is invalid only when at least half the answers are that fast.
No percentile. Scores and quality flags only.
Running this test
Settings you can change
| Setting | Standard | Allowed |
|---|---|---|
| Pictures or words | Pictures | Pictures, Words |
Yearly plans can change settings within these limits. Free and pay-as-you-go studies use the standard settings.
Need a setting this test does not have?
Devices and languages
- Untimed test: runs on desktops, laptops, tablets and phones. Participant devices
- Languages: English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese.
- Available on: Every plan, including the free plan.
- No percentile. Scores and quality flags only. How norms work
- Recruiting online? See the setup guides for Prolific, Sona Systems, CloudResearch Connect and Qualtrics.
Cite and describe
Participants completed the Free Sorting Task in a web browser (ConductCognition task version 1.0.0, scoring version 1.0.0; Lawless HT, Sheng N, Knoops SSCP, 1995; Chollet S, Lelièvre M, Abdi H, Valentin D, 2011), with the standard settings (pictures or words: Pictures). The main score was groups (number of groups made).
ConductCognition versions: task version 1.0.0, scoring version 1.0.0. Every export records the versions each session used, for a methods section.
- Lawless, H. T., Sheng, N., & Knoops, S. S. C. P. (1995). Multidimensional scaling of sorting data applied to cheese perception. Food Quality and Preference, 6(2), 91-98. https://doi.org/10.1016/0950-3293(95)98553-UFree sorting of a product set, analyzed by multidimensional scaling.
- Chollet, S., Lelièvre, M., Abdi, H., & Valentin, D. (2011). Sort and beer: Everything you wanted to know about the sorting task but did not dare to ask. Food Quality and Preference, 22(6), 507-520. https://doi.org/10.1016/j.foodqual.2011.02.004As many groups as people like, each person's 0 and 1 matrix summed for a map, 9 to 20 products and at least 20 assessors.
- Hubert, L., & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2(1), 193-218. https://doi.org/10.1007/BF01908075The adjusted Rand index for comparing two groupings of the same things.
Source papers
- Chollet S, Lelièvre M, Abdi H, Valentin D (2011). Sort and beer: Everything you wanted to know about the sorting task but did not dare to ask. Food Quality and Preference, 22(6):507-520.Paradigm source, Adults
Questions
How long does the Free Sorting Task take?
About 3 to 6 minutes, as listed in the test library.
What device does a participant need?
Untimed test: runs on desktops, laptops, tablets and phones. The browser must be Chrome, Edge, Firefox or Safari.
What scores does it produce?
Groups, Kind Agreement, Groups of One, Largest Group, Things Sorted, Sorting Time and Moves. Each session is also marked valid, needs review or invalid, with the reason.
How widely is the Free Sorting Task used in research?
48 papers from 2021 to 2025 name it in their title or abstract, down 27% compared with 2016 to 2020. That ranks it 439 of 670 test families in the ConductCognition test census (use grade D). Source: ConductCognition test census 0.48.0, counted 12 November 2026 from OpenAlex and ClinicalTrials.gov.
How much reliability evidence does the Free Sorting Task have?
In the published research, 0 papers report its test-retest reliability and 3 registered clinical studies use it as an outcome (evidence grade C in the ConductCognition test census).
Does it report a percentile?
No. Scores and quality flags only.
Which settings can a researcher change?
Pictures or words (Pictures, Words). Settings can be changed on yearly plans; free and pay-as-you-go studies use the standard settings.
Which languages can participants use?
Participants can take this test in English, Spanish, French, German, Italian, Portuguese (Brazil), Chinese (Simplified) and Japanese, with its words in the same language. English is on every plan, and the other languages come with yearly plans. Have a native speaker read the machine-translated text before launch.
Is this a clinical test?
No. ConductCognition is for research use only. It runs, scores and exports the test; clinical interpretation is outside the platform.
Every test: the full test list.
Run this test in your next study
Start free with 10 free participants on any test, then use participant credits or a yearly plan.