How to interpret the Task results tab in tree testing

The Task results tab is where you can pinpoint the most significant results using the task, pietree and task comparison visualizations.

Optimal's tree testing scores are provided for three aspects of participant responses: success, directness, and speed. Successes, failures, and skips are also broken down into direct and indirect, and each Task is given an overall score.

This article explains what each of these things mean.

The success score refers to the percentage of participants who selected a correct destination, regardless of whether or not they had to jump around the tree a few times before doing so.

A success score of around 80% or more is considered a good score for a task. You can see this in the success score bar on the right hand side and divided into green and purple in the pie chart on the left.

The success score is also broken down into direct success and indirect success.

Direct success (green)

This is the number and percentage of participants who navigated to the correct destination via the correct path.

In the example above, you can see that 71% of participants navigated to ‘Make extra payment’ (the correct destination) via the path we defined as correct (Home > Personal loan > Make extra payment). A high direct success percentage means that your labels are clear and make sense.

Indirect success (purple)

This is the number and percentage of participants that landed on the correct destination, but didn’t get there via the correct path.

For example, we can see that 14% of participants ended up on ‘Make extra payment’, but they started down another path before realizing it wasn’t correct and backtracking. If the indirect success percentage is high, it means that participants know more or less what they’re looking for, but they don’t immediately spot a path to get there.

This is the total percentage of participants who took a direct path to land on their nominated destination, i.e. they didn’t move back through the tree at any point.

The higher your directness score, the more confident you can be that your participants were sure of their answers – they knew what you wanted them to find and exactly how (and where) to find it.

Looking at these two scores together can give you good insight into the clarity of your paths and destinations. For example, if you have a high success score but a low directness score, this might mean that your participants know what they’re looking for but are starting down the wrong path to find it.

In this case, it’s a good idea to look deeper into the exact paths they’re starting down and where they’re backtracking. You might need to reiterate on your labeling or task writing.

This is the average amount of time (in seconds) it took participants to complete the task. This is the ‘median’ time and is represented by the line in the middle of the light blue box.

Using the ‘time taken’ measurement is helpful when you’re A/B testing. It can clearly show you if it’s taking participants longer to find a destination in one tree or another.

We exclude participants who took longer than 100 seconds to complete the task because we've found from our data that people who take longer than two minutes to complete a single task have almost certainly left it idle. We remove these outliers to give a better indication of the actual median completion time.

This is the weighted average of your success vs. directness score for each task. Basically, if your overall score is less than 7, you might like to dive a little deeper into the success and directness scores to see how they compare to each other. It could be that the success score is high, but the directness score is low, meaning it’s a good idea to go back over your labeling, paths or tasks.

This is the percentage of participants who clicked skip on a task before they selected a destination.

Direct skip (dark gray)

This is the number and percentage of participants who clicked the ‘skip’ button without even engaging with the tree. This is a good indicator of whether or not your tasks are clear and concise enough. If there are a number of direct skips on a particular task, perhaps people are unsure of what you’re asking of them and don’t feel confident navigating through the tree.

Indirect skip (light gray)

This is the number and percentage of participants who began to navigate through the tree before clicking the ‘skip’ button. This may indicate that once they got into the tree, they felt unsure of where to go next. Perhaps they didn’t see the type of labeling they expected or felt confused by what the tree displayed in comparison to what the task required of them.