Qualitative analysis of NPS data?

[LawrenceMartin]

I spend a couple of hours scanning user responses to open ended questions. There is always a bunch of interesting comments as well as frustrations which I am getting good at not taking too personally:) I manually make notes of opportunities, big pain points and anything else that stands out, but I don’t really have a great quantitative view to show for these.

Do you all have a system for categorizing this data?

[MichaelYoffe]

If you can export the response in Excel, then as you’re reading through them add a column to note the high level category (speed, lacking features, price, customer service…). Then you can filter by that field or create a pivot table to see the most common themes.

[LawrenceMartin]

I am going to try this as soon as I get export access. Thanks!

[AhmadBashir]

Do you have user researchers? They might use a tool called dovetail, I use it its great.

[RohitKumar]

Makes drawing quantitative insights from qualitative data a breeze. Definitely worth the cost.

[LawrenceMartin]

I don’t unfortunately, but I’ll check this out.

[AnaRodriguez]

I banged my head against this for years before I found the following key things made all the difference (aka - my proven formula for improving NPS):

1. Ask the right follow up question. The vast majority of NPS surveys ask the wrong follow up - something like, “What is the reason for your score?” This prompt generates 90% garbage. The right follow up question that generates pure gold is “What could we do to get a better score” or “What could we do to make you more likely to recommend us?”
2. Focus on the respondents that matter. Assuming your goal is to increase NPS (vs. identify new market opportunities or other goals), you really only need to spend time on the respondents who score 6,7,8. Because of the way the NPS calculation works, it is both easier and more effective to move a negative to a neutral or a neutral to a positive (ie. moving a 1 to a 5 feels like a huge victory but makes NO difference in your NPS score not to mention it’s insanely difficult to do). Respondents who score a 6, 7, 8 are on the fence and it won’t take a huge improvement to get them to move.
3. Use Affinity Mapping to identify the priority opportunities. Affinity mapping is similar to the concept of creating categories mentioned by others but is more sophisticated and provides clearer direction.

Hope that helps you be more productive and effective. If you don’t have the time/influence to change #1, you can start immediately with 2 and 3 until you’re able to get the follow up prompt changed.

[LawrenceMartin]

The goal is to both increase NPS as well as identify new opportunities. I am going to try affinity mapping for sure. Thanks for the tips!

[RobMartin]

I normally create additional columns for each reoccurring theme. Eg, columns for Speed, Flexibility, theme n… This allows me add multiple theme per comment and sum the column to see total feedback for a theme, via pivots or simple column sum.

Along with that, I also like to see some customer attributes. Eg, region, tenure with us, MRR, frequency of use, customer tier etc. This allows me to understand feedback themes by different types of customers. Not all feedback is critical, some will be more important eg, new customers complaint about theme X leading to higher churn for example.

[LawrenceMartin]

I didn’t think of customer attributes, but that can be useful info for my product. Thanks!

[MarcoSilva]

If you have data scientists in your team, it might be worth getting them to extract topics from the open ended responses. You can then get the average promoter score for each topic. If you don’t have analysts or don’t have that much data, you can manually code them (by entering columns and entering a 1 if they meet the criteria, like “too expensive” would go under the “price” column).

This could get you insights like “28% of users mentioned that loading times were a problem. They tended to have substantially lower NPS scores, averaging 4.7 compared to 7.7 for everyone else”