May 8, 2012

Now that I have made Shakespeare turn over in his grave and have your attention…

With technology comes challenges and changes.  According to , as of January 2012, 19 percent of U.S. adults age 18+ own a tablet computer, and as of February 2012, 46 percent own a smartphone.  Apple’s iPad and iPhone are leaders in their respective categories with 61 percent of tablet owners having an iPad and 42 percent of smartphone owners having an iPhone.  Generally, penetration numbers are similar or even higher in other developed countries while being slightly lower in emerging countries. 

Apple made the decision not to support Flash technology and other smaller players also are starting to drop it.  Even when devices do support Flash, many simply are not powerful enough to run it properly and the user experience is extremely poor.  Many companies are now creating non-Flash websites and surveys.  Kantar does utilize Flash technology for some questions in surveys (e.g., dynamic grids, drag and drop) and currently is working to transition our survey tools to HTML.

In the meantime, it is important to understand the trade offs involved in using Flash in surveys.  Generally, we are seeing less than 5 percent of respondents not completing surveys because they don’t have Flash.  However, this number has been growing over the last year.   A year ago we usually saw less than 1 percent loss, but now we are regularly seeing losses of 3 percent.

The use of Flash can impact sample representivity because certain respondents may not be able to view the Flash components.  According to Pew, U.S. tablet owners are younger with higher education and higher income, so we could be systematically excluding key demographic groups from surveys.  The fast growth of the iPad in particular means more and more respondents may be trying to take surveys via their iPads thus increasing the odds of impacting the sample representivity.

One option to eliminate the risk is not to use Flash in surveys.  There are certainly other ways besides Flash to make surveys more engaging.  However, excluding the Flash component or substituting a non-Flash version may not always be an option.  For example, Flash may still be necessary to play video.  There are a number of things that can be done to mitigate the impact of Flash. 

First, it is imperative to screen for Flash technology and/or specific device at the beginning of the survey.  This allows us to screen out those who don’t have Flash or don’t have a device that is powerful enough to display Flash properly.  If they don’t have Flash or an appropriate device we can encourage them to download it or switch to another device that allows Flash to be used.  Note, that in order to change devices we need to allow respondents to restart the survey.  Some clients don’t allow restarts. 

Another option if no Flash is detected is to present a non-Flash version to these respondents.  The risk here is that two very different question formats will be presented, which could create bias. 

Clearly, to Flash or not to Flash needs to be considered in the context of the research objectives and the potential risk in using or not using Flash.

Category:Survey Best Practices
Posted on May 8, 2012



Back to Home >>