April 18, 2011

When designing survey questionnaires, we market researchers must constantly remind ourselves to “walk a mile in the respondent’s shoes” – that is, think about how our questions will be viewed and answered by survey respondents.

Over the years, we’ve edited down numerous survey questions that overexplain a straightforward concept. To keep survey questions short and to the point, ask yourself a simple question – “Can I post this entire survey question in one Twitter post?” Generally, if the answer is “no,” you need to shorten and simplify.

The laws of usability place a premium on making it easier for panelists to read, absorb, and process survey information. If a question requires more than 140 characters to impart and elicit information, it is too long.

The following is a great example of a question that fails the test.

Now, we would like you to think about automobiles. We are referring to all kinds of automobiles: sedans as well as compact cars, full size (This is where Twitter would cut off the question) SUV’s as well as small SUV’s, and pickup trucks as well as mini-vans and full size vans. When thinking about all these kinds of automobiles, how many miles per day do you drive?

If you drive more during weekends than during the remaining days of the week, or if several weeks sometimes go by where you do not drive, we would like you to estimate an average.

How many miles per day do you drive?

I had a chance to discuss the Twitter Test concept, one of many survey questionnaire improvement suggestions, recently when I presented to the Online Research Methods event hosted by Research Magazine. In the course of my talk, I reminded my peers that we are at the mercy of our survey respondents; they are not a limitless resource and that we must keep them engaged so they’ll not only complete a single survey, but return to us in the future to do more.

As researchers we should always distill information and avoid using four or five words to say what can be expressed in two. Imposing length limits compels us to be creative and scrutinize our questions to ensure we maximize the quality of the characters we’re allotted.

Did this post pass the Twitter Test? Not quite, but I hope this example of panelist engagement resonates. Limiting characters is good for the respondent, it’s good for the panel, and it’s good for the final data.

Category:Panel Quality, Panel Retention, Survey Best Practices
Posted on April 18, 2011



Back to Home >>