User Interviews: a Starting Guide

Put yourself on a path to conducting effective and valuable user interviews

The best user interviews don't feel like a Q&A session, but a fluid conversation.

If you think you're proactive now and you're not doing user interviews, then you'll have a lot to gain by starting even the smallest effort so elicit feedback from those who contribute most to your bread and butter.

Be warned that if you ignore this tactic, you're going to soon find yourself wondering why you're behind your competitors, where you went wrong, and reacting to a situation. If you think user interviews are for software only, are a waste of money, are going to tell you something you already know, and are an expensive waste, then don't waste your time reading any further.

Analytical tools that measure traffic and "engagement" (okay, I'm talking about Google Analytics, primarily) can really only do so much. They are great at telling you what people are doing and that they are indeed doing stuff on your website. Pat yourself on the back for generating the traffic, but now ask yourself, are you using it to effectively turn those visitors into customers?

Trying to diagnose things like bounce rates, conversion rates, and low engagement, overall, can be difficult using Google Analytics on its own. As most experts attest, you often need to export that quantitative data into a data management and visualization tool if you want to make the most sense of it.

Conducting user interviews is a great way to find answers to those questions without having to take a scientific method-approach towards laborious data analysis.

In the years I've spent moderating focus groups, conducting 1:1 in-depth market research interviews, and working with clients of all sorts, I've found that anyone can benefit from the fundamentals of "user testing", which is a process that, in my mind at least, falls into three key parts: (1) Defining the problem and what you need to solve; (2) Recruiting the right participants from your audience; and (3) Interviewing - not as easy as it sounds.

Step 1: Defining the Problem

What is it that you're trying to solve? I'm sure you're thought of this before you even started thinking about doing some user interviews. It's not so obvious.

Think about this: What is the problem you need to solve as it relates to actions your users are (or are not) taking? You can categorize these actions in any number of ways, but you need to be specific. Remind yourself what the true definition of a goal is: it's specific, attainable, relevant, has a timeline, and a strategy or an action plan to get there. Simply defining "completion my action plan" or marketing the completion of your strategy is a short-sighted way of defining goals. You can go into a big rabbit hole with setting goals by itself, but for our sake, start simple and work with what you've got (especially if you're starting from scratch.)

A common issue most clients of mine will bring up is that they're challenged to solve something like, "Why aren't people signing up for a free trial when it's our page with top 5 traffic metrics?", which is a good starting point, but far too broad to use as a user interview question. Go through an exercise where you write ALL of the questions you're challenged to answer down. Categorize them into problem areas and use those core problem areas to diagnose the big picture.

Ideally, you want to start off your user interview journey with a specific question to solve, for example, "Why do half of free trial sign-ups from Facebook Ads end up NOT upgrading to the full version?" This kind of problem-based starting point leads you to develop open-ended questions that elicit participants to engage in problem solving and provide their own ideas around potential solutions.

Step 2: User Recruitment

In keeping with our simple approach, let's ignore the idea of an "ideal sample size" and assume you know your audience and you know who you want to talk to. What do you do with them? How do you maximize the time you spend talking to them and, in effect, maximize the improvements that the interview feedback has to bring to you? Place a strong balance on the positives and negatives as you develop your line of questioning (just whatever you do, NEVER start or end with a negative question.)

Following the question we're trying to solve at the end of Step 1, we'd know we should recruit participants who have converted on the free trial and signed up as well as those who converted on the free trial and never signed up for the full version. You need to see why people decided to make their decision and what factors were involved. Were they the only person involved in the decision making process? Was it even their sole decision to make?

These may sound obvious, but are often overlooked, and any insistence to the extent they don't need to be asked - either by yourself or by a client - unless they have documentation of recently doing it before, is a strong indicator that something is wrong and they'd sooner operate in naive bliss than face reality and solve a potentially difficult problem.

How do I recruit for user interviews?

Reaching out directly to your customer and prospect base, either by email, or phone - with a personalized message - is the most effective way of engaging participants who will provide meaningful feedback and be engaged during the interview. Every industry is different as is each customer base, but in general, if you send an email campaign asking for participants, a 5-10% response rate is decent. Phone-driven inquiries tend perform better, although be sure the phone calls are not time wasted spent trying to reach people who would otherwise be more readily reached online. Find your ideal participants, and if in some cases, tailor your message and approach to the platform they use to connect to their audience. You're more likely to stand out.

Whenever you do your recruiting, bear in mind there are a few things you'll want to highlight:

  • Give kudos -  "I saw your completed your free trial of BusinessProblemSolver 5.0. Congrats on getting everything set up!" (Exclamation point not necessary IRL.)
  • Incentive -  "Are you interested in extending your free trial, with all features unlocked, for another 30 days? We are conducting interviews with a select group of your customers and would value your feedback." (Note: Unless you have a really good service that people really want to pay for and use, the incentive is going to work much better if it's an Amazon or PayPal gift card. "Cash is king" is a phrase that hasn't died out for a reason.)
  • Ask explicitly "We want to learn how to get people to sign up for BPS 5.0's full version. Do you have time for a 30 minute call this week?" A tool like YouCanBookMe can be helpful in scheduling participant and enabling them to sign up for time slots on their own. However, these kinds of links have no place in your email signature. If you happen to work in sales and use this, consider the idea for a moment that, as a sales person, it's your job to set the appointment and when you tell people to click a link and choose a time, you're in effect saying, "Do my job for me." Don't be that guy (or girl.)

What's the right amount of people to interview?

No matter your company size, audience, etc., if you have started this off by setting a specific question to answer in Step 1, you can typically perform 6-8 interviews to gain enough insight where you can start to develop trends, patterns, shared feedback points, and also begin forming hypotheses for optimization. At minimum, each interview should really be 30 minutes at minimum, but consider this - most people are just going to have started to feel comfortable being more candid with you by the end of that 30 minute period.

If you set the expectation of a 45 minute interview, schedule 60 minutes just in case you go over, you'll often find a very engaged participant who is happy to go beyond the 45 minutes that were originally planned. By this point, if you've been interviewing effectively, they should be pretty chatty by now and sharing a lot of valuable information. Getting your questions answered is important, but bear in mind, when you keep people taking, they'll often reveal much more than you set out to ask about... for better or worse! Be sure you set the appropriate expectations and compensate your participants fairly. They will be more than happy to share constructive feedback.

Step 3: Conduct the Interview

First-time interviewers often forget something invaluable about interviews that they're paying someone to participate in: the interviewer owns the conversation. This does NOT mean you should dominate it - your job is to listen. You are paying for experts' time. Don't be stubborn and think your line of questioning, especially your first time, is going to be right. First time doing this? Budget some money for practice interviews and real user interviews where you might get some value from the participant, but where it's more about you honing your abilities as an interviewer and asking questions effectively.

Before you start, you need to develop a question guide. Google this to your heart's content. There are so many templates out there that it's pointless for me to link to just one. Look at them for inspiration, but continue on your own path and start by defining a few simple things that you need to determine in each individual interview (again, following our free trial example):

  • What pain points were they hoping to solve when they signed up for the free trial?
  • What did they expect out of the free trial? Were those expectations met? If yes, in what ways? If no, why not and what was left to be desired?
  • What big goal are they working to accomplish? How did they envision this solution enabling them to achieve that goal?
  • What is their existing solution for this problem? What have they used before successfully or unsuccessfully? Stay within recent, relevant timeframes. One year to 18 months for software products, for example. Consider your product's lifecycle duration and that of your competitors.
  • What motivated them to sign up for your product over competing solutions?
  • ... and don't forget, you're a bad researcher if at the end of the interview you don't ask, "Is there anything I didn't ask that I should have?" - This can really open things up for some people!

Those questions will provide the basis for a lot of substantial and valuable conversation. It's tempting to ask questions during the interview such as, "Why didn't you just pay for the full version?" However, you've got more to learn by focusing on the motivations that drove them to your solution.

Take feverish notes during the interview. Don't be afraid to go off-script if it feels right and if the person is revealing some valuable information that you've not yet heard before. Choose a transcription service (I use Rev, it's cheap and you get what you pay for, but does the job) so that you've got a written record of the interview and something to reconcile your notes against. In reviewing transcriptions, I invariably find one or two things I completely missed in my notes, so as time consuming and unattractive a prospect it may be, take the time to review them. If you have someone you can delegate some of these responsibilities to, definitely do so. It's tough doing this as a one-person-show. Getting help uploading and managing transcriptions, or booking participants, from something like User Interviews for example, are tasks you can easily outsource and delegate internally or to a virtual assistant.

Interviews Done. Analysis Time.

Okay, you've spent a lot of energy planning for the interviews, conducting them, compiling responses, and now you feel spent. You also feel it might not really be needed to organize all of this stuff because, hey, you spent all this time actually doing it, and it's so fresh in your head, right?

You owe it to yourself to organize this into an interpretable and actionable document. Why did you go through this in the first place if you're not going to complete the final piece? These kinds of interviews are full of information that anyone in any department of a company can benefit from. Make these interviews public to your company (to the extent it makes sense and won't ruin morale). Take the highlights and ask for consent to use them in marketing materials as testimonials and ways to boost credibility. Better yet, leverage the quantitate and qualitative feedback you received into actionable improvement iterations that your team can work towards accomplishing. Organize the feedback into actionable items, assign them to the right team with a deadline, and close the communication loop with the interviewer by validating the implementation of their feedback.

Slowly, these kinds of genuine and organic interactions fuel long-term relationships and will create your company's best evangelists who drive your word-of-mouth business.

Remember that interviews are something you should continually revisit. Make it a point to engage others in your company who are stakeholders in this area. If things seem to stop working, go out and talk to more people. Expand your interview base, expand your approach, and don't be afraid of asking questions no matter what the answer may be. Taking a professional, diagnostic approach keeps you judgement-free and eases the interviewee into more candor. You'll always have something to gain by getting out of the building and having genuine conversations with your customers - not social media conversations - real conversations where you, gasp, actually might meet in person 🙂