The way your questions are written can affect the quality of the responses you receive, so it pays to spend a lot of time fine tuning them when you design your study. Before you begin, think about the types of questions in your study, the tone and wording, the kind of responses you provide participants, and the order in which your questions are listed.
Here are some tips to help you get started with your Questions study.
Pick the right questions for your research
The types of questions you use for your study will largely depend on your research goals. Are you looking for feedback and opinions to improve a product or service? Or perhaps you’re after more stat-heavy data to find out more about your participants.
Open ended questions are great for gaining an insight into what your participants think and feel about certain topics. Example, “What do you think we could do to improve your experience with our support team?”
On the other hand, closed questions are useful for collecting information for things like demographics and other data.
Example: “Which type of device do you prefer to browse the internet with?”
- Desktop computer
Make your questions easy to understand
To capture quality responses from your participants, you need to make sure they know what you’re asking them. Keep your questions simple — try not to pack more than one action into a single question or else it can become too complex. For example, the following question is too packed: “How useful do you find our FAQ page and product landing pages?”. These are two different areas of the site and your participants might have different thoughts on each section, making it hard for them to answer. Instead of stuffing these two actions into one single question, it’s better to split them into two separate questions.
Try to keep your questions free of jargon to prevent any confusion. If you absolutely have to use some jargony terms, try to include a small explanation of what they mean. While some of your participants might know what the word means, there’s a chance that some other participants or people reviewing your results may not.
Keep your questions balanced
As previously mentioned, the way your questions are crafted has a huge impact on the results you receive. This means your questions can’t lead your participants to a certain answer or you’ll end up with biased data.
The following is an example of a leading question:
“The majority of people in New Zealand buy their groceries online. Do you also buy your groceries online?”
This is a leading question because it states that most people in New Zealand buy their groceries online, which hints that it’s the ‘correct’ answer. Instead, this question could simply be cut down to “Do you buy your groceries online?”
The same applies for loaded words (words that invoke strong feelings, whether positive or negative). For example, “We think our customer success team is really awesome. How awesome do you think they are?”.
Provide the right kinds of options
If you’re providing questions with dropdown, radio button or checkbox responses, you’ll also need to give your participants options to select from too. To get quality data, make sure your responses are clear and don’t overlap one another.
Take the following question as an example:
If a participant had 5 members in their team, they’d be quite confused as to which option to select, and in the end it might skew your data.
Designing your survey
Think about the order
The way in which your questions are ordered can affect your results. In fact, you can even bias your results due to the order of your questions. Participants can be primed into thinking about an answer to a previous question while they’re answering another question.
For example, “What are some of the deadliest food allergies you know of?” then the following question could be “Do you believe peanuts and peanut products should be banned from all schools?”. The first question makes participants think of some of the deadliest food allergies they know (hopefully bringing peanuts to mind), while the second question reminds them of the potentially deadly association peanuts can have.
At the same time, it makes sense to order your questions in a particular way with a logical flow so that your participants can follow them easily. This is handy if you’re researching a particular process or something else that has a chronological order to it. For example, you could be researching the online shopping experience on your website:
- How often do you visit our site?
- Did you find what you were looking for on our site?
- If you did not find what you wanted, can you describe what you were looking for?
- Describe your experience with the checkout process.
- Did you receive the item you ordered? Etc.
Within the tool, you can opt to randomize the order of your questions on the Questionnaire tab. Additionally, you can also drag and drop your questions to change up the order.