June 4, 2012
Is your sample size sufficient?
By Susan Frede, VP, Research
Unfortunately, researchers often forget statistical best practices when it comes to sample size. With smaller sample sizes there is more variability in the data and a single respondent can have a large impact on the results. In addition, with smaller sample sizes smaller differences may be missed. All of this can lead to inaccurate business decisions.
As sample size decreases, the confidence interval around a measure increases. A confidence interval is the range in which the true population value falls. For example, the chart below calculates confidence intervals using a proportion of 50% and a confidence level of 95%. At a sample size of 100 the confidence interval is +/-9.8%, meaning that the true population number lies between 40.2% and 59.8%. This is a wide range and a value of 40% vs. 60% could make a big difference in a business decision. Compare this to the +/-4.4% confidence interval at a sample size of 500, which means the true population number lies between 45.6% and 54.4%. Clearly, there is more precision in the bigger sample size and we can feel better about our business decisions.
Sample size also plays a role in statistical testing. With smaller sample sizes it takes bigger differences to be statistically significant. For example, at a sample size of 400 an 8-point difference is significant at the 95% confidence level. While at a sample size of 100, it takes a 15-point difference to be significant at the 95% confidence level. If you are trying to pick a winning product idea, it will take a large difference to be significant at a small sample size.
It is extremely important at the design stage of a research project to make sure you will have sufficient sample size in total as well as for key target groups. In addition, a bigger sample size may be called for when there is more risk involved in the business decision (e.g., major capital investment, change in product formulation, etc.). Lightspeed Research staff can help address concerns and questions about the appropriate sample size for your project.
Category:Data Quality, Research on Research, Survey Best Practices
Posted on June 4, 2012
Back to Home >>
Leave a Reply