Add qual colour, nuance and depth to your quant research
Photo by ismail yazıcı on UnsplashAnswers to open-ended questions in online surveys can produce valuable insights in respondents’ own words, which add colour and a human voice to the results. However, often they can be disappointing, repetitive or appear throw away or low engagement.
In our years of mastering digital research, we have evolved various ways to get the best possible answers from open ended questions, without necessarily adding any time or cost to quant surveys. But first , let’s consider when and where things can tend to fall down.
Why open-ended answers are often disappointing
One major let-down, especially with quant surveys, is that many respondents race through questionnaires, not stopping to think more deeply on open-ended questions. Often, when completed at speed, they may not give the time needed to understand what they are being asked, or even be able to answer.
So what can be done to improve things? Here are five tried-and-tested suggestions.
Review the soft launch results
If you read the first few answers to your open-ended questions you will be able to tell if respondents have misunderstood what is expected of them. You can then re-word any such questions and potentially run another swift soft launch (rather than going straight into full fieldwork), and review the answers for ‘gettability’.
Add a “Don’t know” checker
It is possible to add a control into an open-ended question, such that if a respondent writes “Don’t know”, “Not sure”, “None”, etc… then a prompt is shown asking them to think further about their answer. This prompt can even re-phrase the question for them.
Avoid using the word “Why” in the question
If an open-ended question is phrased as “Why did you….?” this can imply some underlying judgement within the question. It is better to ask something like “What were your main reasons for…”, or some other wording that does not suggest overt judgement.
Include a minimum word count
It is possible to put in a control into open-ended questions which says “Please write at least xx words in your answer”. Not only does this ensure greater depth of response but it will often mean that respondents become more expansive and exploratory in their responses.
Use time-controls
Another possibility is to prevent respondents from jumping to the next question before a certain amount of time has elapsed. So, an open-ended question might say “Please tell us what you think about xyz. To give you time to think about this we have hidden the “Continue” button for x seconds”.
In summary
All of these techniques are tried and honed by us. Testament to it working is that the depth and quality of our verbatims from open-ended questioning in quant surveys is often praised by our clients. Some of these methods require the survey software to have particular capabilities, but others do not. Either way, ‘thinking in the respondent’s shoes’ and building in natural quality and sense checks and ‘pacemakers’ are great foundations for ensuring greater insight and context from online surveys. Get in touch if you would like to find out more.