Building the MVP for a B2B SaaS product while handling only 6–8 interviews. My manager suggested that I record them so I could learn from them. This is helpful for valuable insights but requires a lot of work. unable to use a tool since they spoke in their native language.
Here is my (very broad) approach, which assumes that your usability evaluations involve asking users to complete specified activities rather than doing a “free form” study.
I make a matrix for each participant with the following tasks listed in columns and rows:
Could one accomplish the task? (Objective yes or no)
How firmly did they carry out the task? (Subjective based on your perspective, but you could also balance this with more objective measure like measuring how long it took)
Soundbites: direct quotes from the participant that are objective and word for word
Observations: Your unbiased notes on what you observed (they don’t have to be comprehensive; just include anything that stands out as significant).
Your own subjective notes about what transpired and why the person encountered difficulty (I like to push myself for more than one hypothesis rather than narrowing down)
Ideas: Any suggestions for specific adjustments that could solve issues if the person reattempted the assignment (and tie back to hypotheses)
If nothing leaps out for some people, there might be blank cells, and that’s okay.
Then combine this data for all participants and concentrate on areas with a high degree of similarity.
People who take a more general approach to insights often introduce a lot of their own bias into their observations. I prefer to take a very structured approach and purposefully separate out objective, subjective, and ideas-for-changes.
Dovetail is what I use as a UX researcher to analyse interviews. greatly reduces time. You can highlight the transcription after the tool has finished transcribing the video recordings. To aid with the native language issue, you can also add in your own terms that Dovetail will remember while transcribing interviews. After that, you can annotate and highlight your notes and transcriptions, compare them to notes from other interviews, and draw conclusions. For reference, the insights will include the dialogue, notes, and video clips.
There are both short-term and long-term strategies for interviews. You are correct, though, because conducting, transcribing, and regurgitating interviews takes time. Sadly, it is exactly the way that particular beast is.
You are creating a body of evidence for patterns and analysis, just like any other data point. Therefore, you need to be aware that your B2B product workflow would involve conducting more than one round of 6–8 interviews. It’s more likely that you’ll perform three sets of eight repetitions during the start, middle, and end of a large release.
Create an input matrix for the critical intersections in your product workflow. As a result, the rows would be the user list and the columns would be observation instances. Use the same table inputs as much as possible to establish a benchmark for your progress over time.
So, make it collaborative. Encourage a designer or engineer on your team to accompany you to the interview so they can take notes. It assists in the gathering of insights and the development of consumer empathy. Win, win.
Restrict expectations. The usual procedure is to have someone take notes. Does not really matter who. Check Nielsen-Norman. If they don’t provide someone, you’ll have to analyse every recording, and the analysis will take twice as long. Plus you might have overlooked something.
Give customers the option of good, fast, and quality or lousy, slow, and inexpensive.