Understand and standardize categories

This article describes:

In an open card sort, the table displays all categories created by your participants, and the cards placed in each category. In a hybrid card sort, you'll see your predefined categories (with the blue stripe), the ones your participants have created, and the cards in each.

Below, you can see that "Arts and culture" is a predefined category, and the two "Careers" categories are ones participants have created.

In a closed card sort, the table lists your pre-set categories, the number of unique cards placed in the category, the cards themselves, and the number of participants who placed each card in that category.

Standardizing categories means merging similar categories together to turn them into one category.

When you allow participants to come up with their own category names, there's a good chance you'll see the similar labels that have variations in wording, spelling, capitalizations, and so on. Furthermore, when you take a closer look at the cards, you can often deduce that different participants mean exactly the same thing by their category labels.

It's a good idea to standardize categories before exploring your results in depth. It prevents cross-over and reduces the complexity of the analysis, particularly if you have a large number of completed card sorts.

Before you eye up a bunch of similar-looking labels and standardize straight away, it's important to look at the similarities between the categories in more depth.

Here's a simple process to follow.

The categories table is initially arranged alphabetically, so you'll be able to see similar categories next to each other by just glancing through the table. You can also use search to display only categories that contain keywords.

In the example below, you can see that there are two categories we could standardize – “City info” and “City information” – as both include some of the same cards.

We might be inclined to select all seven categories containing the word 'America', and hit 'Standardize' with a sense of achievement and delight. But first, we need to make sure the distinct categories we're about to merge are similar enough to become one.

Two options for approaching this include looking more closely at the category labels and the cards in each category, and checking the agreement score after we've created a standardized category. The first is more subjective, and involves your own insights and decisions, and the second gives you a hard figure. Both approaches complement the other.

The agreement score tells you the agreement level between included participants on the cards that belong in each category. Beside the agreement score, you can see the number of participants included in that score.

A perfect agreement score is "—", which is what you'll see before standardizing as each category has been created by one person and therefore an agreement score does not make sense in that situation. People tend to agree with themselves!

Once you standardize a category, check the agreement score to get an objective assessment of how similar the groupings are. In the example below, we can see that the agreement score for standardizing “City info” and “City information” is 68%.

Any agreement score of over 60% generally means you’ll find it useful to keep this category standardized for the rest of your analysis.

In the example below, we can see that the agreement score is only 31%.

When you see an agreement score that's low, it means that participants are probably thinking about the categories in different ways, or may have placed cards in categories that don't seem to make sense. At this point, we could reassess our merged categories.

However, when we look at the cards, we can still see something useful. The cards are displayed in order of occurrence, and we can see that the top four cards have a high frequency (meaning they were placed into this category X amount of times) and an average position of between 1.9 and 3.1 (this is where the card was placed in relation to the other cards, i.e. was it at the bottom of the list or the top?) This gives us confidence that grouping these cards together is a good idea.

Your standardized categories will be displayed in the Standardization grid, so you can clearly see how often each card appears in the category you've created.

The grid shows you, at a high level, the number of times a particular card appears in a standardized category.

The Categories table for a closed card sort tells you how many times a card was sorted into each given category. You’ll also find out how many unique cards were sorted into each category, with fewer unique cards meaning higher agreement among participants.

You’ll approach closed card sorting results with questions like:

With the closed card sort categories results, you could:

The example below shows us that there is high agreement for the top cards in this category. So it’s clear that the top five topics belong in the same category.

The “Community support and resources” category (below) had 24 cards sorted into it – the highest number of unique cards in one category. This suggests that the label itself was too broad, and therefore too ambiguous for our website.