Understand and standardize categories

This article describes:

In an open card sort, the table displays all categories created by your participants, and the cards placed in each category. The 2 columns on the right (agreement and number of participants) become relevant when you standardize categories, which we talk about below.

For example, we can see here that one participant created the category 'International Tourist East' containing 6 cards.

In a closed card sort, the table lists your pre-set categories, the number of unique cards placed in the category, the cards themselves, and the number of participants who placed each card in that category.

For example, we can see that 42 unique cards were placed in the 'Not important at all' category. And in particular, the card 'has a dj' was put in this category by 37 participants.

And in a hybrid card sort (in which you have pre-set categories and allow participants to create categories of their own) the table displays what you see in open and closed sorts: your categories and the participants' categories, the unique cards placed in each category, the number of participants, and the agreement between participants.

Standardizing categories means merging similar categories together to turn them into one category.

When you allow participants to come up with their own category names, there's a good chance you'll see the similar labels that have variations in wording, spelling, capitalizations, and so on. Furthermore, when you take a closer look at the cards, you can often deduce that different participants mean exactly the same thing by their category labels.

It's a good idea to standardize categories before exploring your results in depth. It prevents cross-over and reduces the complexity of the analysis, particularly if you have a large number of completed card sorts (like in a recent study we ran, which had 195 completed card sorts and over 1,400 suggested categories!).

Before you eye up a bunch of similar-looking labels and standardize straight away, it's important to look at the similarities between the categories in more depth.

Here's a simple process to follow.

The categories table is initially arranged alphabetically, so you'll be able to see similar categories next to each other by just glancing through the table. You can also use search to display only categories that contain keywords.

In this example, we've searched for the keyword 'America', and have found that 7 participants included that word in a category label. This could make it a great candidate for standardization.

We might be inclined to select all 7 categories containing the word 'America', and hit 'Standardize' with a sense of achievement and delight. But first, we need to make sure the distinct categories we're about to merge are similar enough to become one.

Two options for approaching this include looking more closely at the category labels and the cards in each category, and checking the agreement score after we've created a standardized category. The first is more subjective, and involves your own insights and decisions, and the second gives you a hard figure. Both approaches complement the other.

For each category, you can look at the cards that have been sorted in each. You can opt to view all the cards in each category by clicking on the 'Show 'X' more' link.

When we look at all of the sorted cards in each category, we can see 5 of our 'America' categories contain at least 2 of the same cards as the others. This gives a good indication that the participants had similar intentions when creating those groupings, and we could consider merging them into one category.

However, when we look again at the category labels, we see that one is actually 'America (North & South)', and that it includes countries also in South America. Another category only contains 'Mexico', and another only 'Istanbul'. These groupings suggest a different intention than the others, so we could probably exclude them.

Therefore, we could create a standardized category called 'America' containing the 4 categories with similar cards.

The agreement score tells you the agreement level between included participants on the cards that belong in each category. Beside the agreement score, you can see the number of participants included in that score.

A perfect agreement score is "—", which is what you'll see before standardizing as each category has been created by one person and therefore an agreement score does not make sense in that situation. People tend to agree with themselves!

Once you standardize a category, check the agreement score to get an objective assessment of how similar the groupings are. Agreement scores from .8 and over means that 80% of participants in the new category agree with the grouping. This is a good standard.

So, what agreement score do we get for our 'America' category? Only 0.51.

When you see an agreement score this low, participants are probably thinking about the categories in different ways, or may have placed cards in categories that don't seem to make sense. At this point, we could reassess our merged categories.

However, when we look at the cards, we can still see something useful. The cards are displayed in order of occurrence, and we can see that the top 4 cards have perfect 1.0 agreement among participants (included in this category 4/4 times). Furthermore, the next 3 cards have an agreement of .75, which can also give us confidence that grouping these cards together is a good idea.

Your standardized categories will be displayed in the Standardization grid, so you can clearly see how often each card appears in the category you've created.