Askin’ Ain’t Easy: Tips on asking the right questions during user interviews

Why interviews are crucial

Whether you’re a product manager, designer, or engineer we (should) all agree that empathizing with our users is a fundamental part of designing a successful product. Learning more about our users can happen in many ways: click through rates, emails, customer service phone calls, A/B tests, etc. But if you want a deeper understanding of what your user thinks and feels you’ll need to focus on qualitative methods like usability tests, ethnography, diary/camera studies or contextual inquiry.
There are a variety of options available, but nearly all qualitative user research hinges on one fundamental technique: interviewing.

Interviewing: the art of asking questions

In the past, I’ve witnessed (and been guilty of facilitating) interviews gone wrong.

Questions like:
1. “Do you like this design?”
2. “Would you pay for this service?”
3. “Do you think this is easy to use?”
4. “Wouldn’t this be great on an iPhone?”

…are doing you and your team a disservice. It’s setting you further back than if you had not conducted an interview at all. Why? The participant didn’t respond to the question — they responded to you asking the question. Leading questions create bad, skewed data that doesn’t represent your users’ thoughts or feelings. Your findings are influencing the product and it’s leading you down a (likely expensive) path that doesn’t serve your users needs.

Your findings are influencing the product and it’s leading you down a (likely expensive) path that doesn’t serve your users needs.
To get the heart of your user, you need to practice nondirected interviews. Nondirected interviews involve open-ended questions which intend on eliciting honest feedback from the participant. If practiced correctly, it reduces the risk of the moderator influencing the participant’s answers.

Don’t wing it: write a guide

Asking opened-ended, non-leading questions is not an easy task. When you think about our day-to-day conversations, it’s filled with voicing our opinions, biases and attempts to get the listener to see things our way. One way to help filter out your natural tendencies is to write a guide. With a well-written interview guide and a little practice, you can conduct a thoughtful interview and collect great insight. Below are tips I use when I write a new guide.
Note: to ensure your guide is in a good place, it’s helpful to write the first draft and share it with a partner.

Interview Tips

Avoid assumptions

When writing your questions don’t assume a user believes or knows certain things.

Don’t: What benefits do you get by using Trunk Club? This assumes there’s a benefit to the user.

Do: How would you describe your experience with Trunk Club? When you ask a question with assumptions about a user’s feelings, they’ll focus on giving you answers that fall in line with the verb used. When you broaden the question and remove the assumption (in this case, ‘benefits’), it gives the user the ability to describe the experience in their words.

Ask open-ended questions

When you ask binary or close-ended questions, people will pick something even if it’s not an option they would have considered.

Don’t: Do you like this page? This can be answered with a simple yes or no.

Do: Tell me what you think of this page. In some instances, it’s impossible to ask a question that can’t be answered with a simple yes or no. A tactic to get more in-depth answers is to ask “why” after these questions. Using “why” allows the participant to confirm or deny their assumptions and to dig into what led them down the thought process that inspired the answer they gave in the first place.

Avoid leading questions

If the participant feels you’re looking for a specific answer, or if they can give a wrong answer, you’ve ventured into leading-question-land. A leading question may include the answer in the question, or some sort of indicator of what the right answer may be. Note that body language can play a role in this as well.

Don’t: Of the two designs you looked at, do you think the new design is easier to use? This question implies that the new design is easier to use.

Do: Of the two designs you looked at, which design is easier to use? Why? Avoid having participants predict what they would do in the future
The future is aspirational — humans have a tendency to lie when asked to postulate about it. Asking for what they’ve done in the recent past can give you insight as to what they may do in the future.

One of my favorite analogies is signing up for a gym membership. You plan (or predict) on going 3–4 times a week. But once you sign up, most of us aren’t good about going the number of times we intended.

Don’t: Would you pay $150 for this subscription-box service? Maybe your user would pay $150. It sounds reasonable at the time, but would they go home and do it?

Do: Have you paid for a subscription-box service in the past?

Do: How long ago?

Do: How much was it?

Do: How did you feel about paying $49.99 a month?

Ask one question at a time

Focus on one question at a time. An “and” or an “or” linking two ideas is ambiguous. If it’s a meaty question, it can also cause the participant to forget the second half of what you asked.

A great example of this would be the different types of Trunk Club members. Some members exclusively receive trunks, some exclusively come into our clubhouse and some do both. Instinctively, you would ask a member which of the three options applies to them.

Don’t: Have you received a trunk or come into the clubhouse? If they said yes, which one applies to them? Did their answer mean they received a trunk, come into a clubhouse…or both?
However, if you break the question into two parts, you can get a direct answer and the opportunity elaborate on each answer.

Do: Have you ever received a trunk?

Do: Have you visited the clubhouse?

Follow up with why

Sometimes participants will give you surface level answers. When you ask why it gives you an opportunity to learn more about their beliefs and behaviors. It also allows the participant to dive deeper into their frustrations, challenges and generally ‘vent’ about their experiences. Some researchers use the ‘ask why five times’ rule of thumb. You might find it handy if you’re looking to dig into the root cause of their problem.

Q: Have you paid for a subscription box service in the past?

A: Yes. I’ve used botbox.

Q: Why did you use botbox?

A: I used botbox because I really like robots.

Q: Why do you like robots?

A: I think it makes my life easier.

Q: Why do you need to make your life easier?

A: Because I work a lot of hours and I’m looking to spend quality time with my friends and family.

As you can see in the (silly) example above, asking why a few times allows you to get to the cause of why someone subscribes to botbox. If asking “why” seems redundant, mix it up with the phrase “tell me more about [needing to make your life easier].”

Pause and listen

Momentary silence is your friend when interviewing. When you ask a participant a question, it may take a moment for them to think through the answer. Heck, they may even change their answer entirely once they’re given a moment to think about what they said. In addition, taking a pause and focusing on the participant gives them an opportunity to look back at you and notice that you are listening to everything they’re saying, and that should lead to a more earnest response.

Now go practice!

Writing a guide is not an easy task, and neither is interviewing. It takes practice and gets easier as you interview more users. In the end, interviewing users should not feel like an interview for either of you. Once you get into the “interview-groove”, you’ll be able to jump in, speak to someone, make them feel at ease, and get them comfortable with giving you the most honest and useful feedback they can provide. You’ll learn what kinds of questions need more direction, and what can be a little more ambiguous. Good luck!