I continue to be surprised by how poorly crafted most surveys are. The avoidable mistakes lead to, at best, confused and unactionable market data. At worst, you'll use bad data to justify making all sorts of bad (and expensive, and time-consuming) product and marketing decisions. Below is an example of just one bad question from a survey I received from a travel website. While the survey is geared toward consumers, the lessons are just as valid for B2B tech products. Let's take a look at three mistakes in this question, and how they could've been avoided:
Survey Mistake #1: Mismatched Question and Answer Options
Asking me “how important” each feature is makes me assume you want a rank-order, not a selection of the most important features of a product. The question comes out of the gate confusing the respondent! How to Improve: Understand what you're trying to find out and keep the question and format consistent with that goal.
Survey Mistake #2: No Scale in the Order of Magnitude
In no world would a helipad and a swimming pool hold equal measure in choosing a hotel. This is an "order of magnitude" mistake - putting options together that are wildly more or less important to a product buyer. When you do this, you often get reinforcement that your market values "the basics" and isn't interested in attributes that would differentiate your product from others in the market. This may not be true; it may mean that they expect a swimming pool, but would choose you over other hotels with swimming pools if you had a helipad. How to Improve: Separate ranking of differentiators from a validation of “must have” product features.
Survey Mistake #3: Use of the Dreaded “And”
“Sustainability” and “community investment” are two very different things – what if I value them differently? If I do, you’re going to get bad data from this survey option. Either I'll ignore it because I can't answer the same way for both options, or I'll mentally average two unique scores to give you one - inaccurate - answer. How to Improve: Check every use of “and” to make sure you haven’t munged two questions into one.
To innovate on purpose you have to listen to your market intentionally. You don't have many chances to engage - use them wisely with carefully-crafted surveys that get you actionable insights.
About the Author
Diane Pierson is the Founder and Chief Market Strategist of Innovate on Purpose, a consultancy enabling successful product commercialization for B2B tech companies. Order her book, How to Innovate on Purpose or contact Diane at dpierson@innovateonpurpose.com.
Comments