How to Conduct B2B SaaS Product Feedback Interviews

Customer research · Customer interviews

Every software company has a gap between how its product was designed to be used and how customers use it, and in our experience the size of that gap surprises the people who built the product. Most customers work around a limitation without reporting it. They turn an export that nobody expected them to rely on into their main workflow, or they keep a spreadsheet beside your product to do the one thing it will not do, and in our experience most of them stop mentioning any of this a few months after going live. A product feedback interview is how you find out what they are doing now. The two questions a product team usually reaches for first ask which features the customer would like and how satisfied they are. Both give answers that are hard to act on. Section 5 explains the problem with the first, and the overview explains the problem with the second. This guide replaces both with questions about things the customer has already done.

What are product and service feedback interviews for?

Product feedback interviews show how customers use your product

The purpose of the call is to find out how your product is used inside the customer's operation: which parts they touch daily, which parts they have never opened, what they do in the moments when it stops being useful, and what they have built or bought to cover the gap.

In our experience a round of these produces three things. The first is a list of workarounds. A workaround is a feature request with the business case already attached, because a customer who built something themselves has already decided the problem was worth their own time. The second is a list of what customers pay somebody else for alongside your product, which you can treat either as roadmap candidates or as a list of possible partners. The third is the question customers ask in month two that they should have had answered before they bought. Customers tend to raise it first with your support team, and in our experience the fix usually belongs to marketing or onboarding.

The round gives you a list of candidates. It does not tell you the order to build them in, and the limitations section below explains why.

When should you conduct product and service feedback interviews?

Run feedback interviews once the product is in daily use and a roadmap decision is coming

The first condition is that there is something to interview about. People use the product regularly, and a roadmap or service decision is close enough that what you learn can inform it. Early in a product's life there is nothing to describe, and a discovery interview is the better instrument. After a roadmap is locked, these calls raise expectations you cannot meet.

In our experience the second condition is the one teams miss. Somebody has to have agreed to act on the output. A discovery round produces findings for marketing work you control yourself, and this round produces findings for product and customer success. If neither of them asked for the research and neither is going to read it, you are collecting requests into a drawer. The customers you interviewed will also notice, over the following months, that nothing came of it.

Who should you talk to in a product feedback interview?

Don't talk to the buyers; talk to the people who use the product

Of the five interview types in this series, this is the one built around the daily user. Whoever signed the contract usually cannot tell you what happens in the product on a Tuesday afternoon, and the person who can is often two levels below them and has never been asked.

Recruit across use intensity and not across company size. Three heavy users, three moderate ones and two people who were trained and then stopped will tell you more than eight enthusiasts. In our experience the person who stopped is the most useful of the three groups to hear from, and the hardest to get on a call. Nobody inside the account wants to put them forward.

Ask in every call who else touches the product, and interview at least one of the people they name. People are least reliable about what happens inside their own company, which is why the overview's rules for every call recommend two interviews per account. That holds for how a team uses a product as much as for how it buys one.

Who is this round for?

Decide before you start who receives the findings

Settle this in advance, because it changes which questions you ask. If product owns the output, the useful material is workarounds, missing capability, and the sequence of actions around a specific task, so your follow up questions should ask for the steps the customer took. If customer success owns it, the useful material is where people get stuck, what they were never shown, and how many weeks into the relationship they got stuck. If marketing owns it, you are looking for the distance between what the product is sold as and what it is used for.

Pick one owner and run the round for them. In our experience a guide that tries to serve all three at once produces transcripts that everyone skims and nobody uses.

The questions below are written for a product owner, since in our experience that is the most common case. If marketing owns the round, keep the structure and add three questions to section 3: what did you think the product would do for you when you bought it, what does it do for you now, and if you were describing it to a peer, what would you call it. If you are the founder of a small company and you are product, customer success and the person who sold the deal, choosing an owner does not help. Name the decision the round has to inform, and run it for that decision.

What does a typical product feedback interview look like?

The standard 30 minute format

The call has six sections, built from the structure of a discovery interview. The switch story is cut to two questions. Discovery's single section on value and frustration becomes two sections that together take half the call. The cross sell section narrows to what the customer bought or built elsewhere, and the pricing section is dropped.

The order puts value before frustration. Opening on what is broken tells the customer the call is a complaint session, and in our experience customers stay in that register once they are in it. Seven minutes into a list of failures, the question about the last time the product saved them trouble gets answered grudgingly or not at all, because they are now recalling their experience in the frame the first half of the call set up. Running value first costs nothing. It also tends to produce a more forthcoming interviewee by the time you reach the questions about what they built behind your back.

The structure does cost you one thing. Nobody will give you a usable account of why they chose the product, because section 2 is only two questions deep. If a discovery round is running late and somebody suggests using these calls to cover both, run the discovery round separately.

Section 1 · 3 minutes

Warm up and their operation

Open the way you open any call in this series. Say who you are, that this is a research call and not support or sales, and what happens to the recording. Then establish how the team is organized, since every answer later in the call has to be read against it.

The last question sorts your interviewee into heavy, moderate or lapsed use. Use intensity is the variable you recruited on, and you will read every other answer against it afterward. Write the answer down during the call.

Section 2 · 4 minutes

How they came to it

Ask two questions here and no more. The aim is to place the person in the account's history. The full account of how a customer came to buy belongs in a discovery call.

Somebody who inherited the product gives you different material from somebody who chose it. Having had no part in the decision, they have no stake in it having been the right one, and in our experience they are the most candid interviewees about what the product does badly.

Section 3 · 7 minutes

Value moments

Ask about episodes. The questions that work ask about the last time something happened, and then about what the person did next.

The third question is specific on purpose. Few people can answer "what do you find most valuable" accurately, because the question asks them to rank things they have never compared. Most people can tell you what they opened on Monday and what they did with it afterward, and across eight interviews those answers give you a rough version of the ranking the direct question was trying to get.

Then ask about dependency, which approaches value from the other side.

Both are hypothetical, in a series that argues against hypothetical questions. We keep them because they do not ask the customer to predict a decision of their own. They ask which parts of a product in daily use the customer would notice losing, and a daily user knows that. Pay as much attention to what people would give up on as to what they would replace. The list of things they would give up on tells you which parts of your product they tolerate rather than value, and in our experience it is longer than anyone inside the company expects.

Section 4 · 8 minutes

Frustration and workarounds

At eight minutes this is the longest section, and its material is what the round is run to collect. You are looking for the places where the product stops working and for what the customer did next. People keep working when software fails them, so there is usually something to describe.

Ask the question about maintenance even when the customer has already described the workaround, because its answer tells you what the problem costs them. A workaround costing somebody two hours a month is an irritation and rarely justifies engineering time. One costing two days a month is a budget line that the customer has already chosen to pay. When three customers describe something similar, you have a better argument for engineering time than feature requests give you, since each of them has shown what the problem costs.

Ask about the request that went nowhere as well. It dates the frustration and tells you what the customer believes about your responsiveness.

Section 5 · 5 minutes

What they bought or built elsewhere

What customers pay for alongside your product shows you capability they wanted enough to buy or build. Ask about purchases they have made, since a purchase records a decision already taken.

Do not ask which features they would like. Product teams reach for that question first, and it asks the customer to design software, which is neither their job nor their expertise. Asking what they already pay for elsewhere gets you much the same information with a price attached and a decision already made.

Section 6 · 3 minutes

Wrapping up

Ask who else you should talk to, then ask the open question and stop talking. Ask for permission to quote once the customer has finished.

Count to seven after the second question. In our experience the closing question in this interview type often produces some of the most useful material in the transcript. Frustration with a product somebody uses every day is spread across many small moments, and a question about one specific episode rarely draws it out. A silence after an open question gives the customer room to raise it.

limitations

What a product feedback interview does, and doesn't, tell you

You get the distance between designed use and actual use, the workarounds with their maintenance cost attached, the adjacent spending, and the month two question your marketing should have answered before the sale. You also learn which parts of the product people tolerate and which they value.

The round produces a list of candidates, and it does not rank them. Twelve conversations can easily produce fifteen things worth building and no reliable order to build them in, since an order drawn from twelve conversations mostly reflects which twelve customers answered your email. Ranking needs a survey. This round gives the survey its options, in the form of problems customers raised themselves, which you would otherwise have had to guess at.

The published evidence on customer input and product success shows a positive relationship of limited size. A meta analysis of 233 studies of new product success found that customer input correlated with success at about .20, based on 47 correlations from ten independent samples (Evanschitzky, Eisend, Calantone and Jiang, 2012). The relationship is statistically significant, and a correlation of that size corresponds to about 4 percent of the variation in success. The same analysis found no significant relationship between success and whether the product met customer needs, a factor with six independent samples behind it. The underlying studies measured success in several ways, some subjective and some objective, and the analysis treats customer input as one broad factor without reporting interviews separately, so what it says about this method is indirect.

A second meta analysis is more specific and includes a business to business moderator. Across 39 independent samples, customer participation in new product development, meaning customers taking part in developing the product itself, improved new product financial performance when it happened at the ideation and launch stages. When it happened during development, it slowed time to market, and through that it hurt financial performance. The benefits were larger with business customers than with consumers (Chang and Taylor, 2016). That paper measured customer participation in development and not interviewing, so it does not describe a team doing continuous discovery. It does show that the stage at which customers shape the product makes a difference. We read it as a reason to run this round before a build is scoped and again at launch, and to keep customer input during the build to the questions in sections 3 and 4.

The cost of the round is small enough that a positive association of this size justifies it. It comes to eight to twelve thirty minute calls, the recruiting around them, and a day of synthesis.

Taken together, the two findings suggest using the round to find problems, and leaving the choice of solution and the order of work to the product team. A correlation of about .20 does not support building a roadmap out of customer requests. The calls do produce a set of problems the company did not know its customers had, described in enough detail that somebody can judge whether solving them is tractable.

Between the transcripts and the candidate list there is a step this guide has not yet described, and in our experience it is the step most rounds skip. Tag each episode a customer described with three things: the use intensity of the person who described it, what the workaround costs them in hours a month, and whether anyone else in the round described the same thing. The candidate list is the episodes with a cost attached that more than one person raised. Everything else goes into a holding list for the next survey.

Then take the candidates into whatever process you already use for prioritizing work, and let that process set the order. Where a workaround costs a customer several days a month and three other customers describe something similar, you have a case worth making. Where it appeared once, you have a note to check in the next survey.

What you should watch out for

Dos and don'ts in a product feedback interview

FAQs

Frequently asked questions about product feedback interviews

How is this different from a customer success check in?

They do different jobs, mainly because different people ask the questions. A check in is run by the person who owns the relationship, and it is good at the things that position allows: continuity, usage data on the table, and a conversation the customer expects to have consequences. A feedback interview is run by somebody with nothing to protect. Customers will describe workarounds and frustrations to a researcher that they would not describe to the person whose numbers depend on their renewal. Run both.

Who should run these calls?

Somebody outside the account relationship, for the reason above. A product manager can run them well, provided they can hear criticism of their own work without answering it. In a company too small for that separation, name the conflict at the start of the call, have a second person take the notes, and hold to the rule in the don't card about not answering your own question when the customer pauses.

How many product feedback interviews do you need?

Eight to twelve for a readable round, spread across heavy, moderate and lapsed users. The spread matters more than the number, since eight enthusiasts will agree with each other and with you.

Can you combine this with a discovery interview?

Partially. A few feedback questions fit inside a discovery call, which is why the discovery structure has a value and frustration section of its own. The reverse does not work, because this call cuts the purchase history to two questions and will not produce positioning material.

What do you do with the feature requests that come out of it?

Treat them as candidates, and turn the ones that recur into survey questions so that the ranking comes from a sample. Workarounds get more weight than requests, because a customer who built something themselves has shown what it is worth to them. A workaround that several customers describe, and that costs each of them days a month, is usually enough to act on without waiting for the survey.

How do you get lapsed users on a call?

Ask the account contact for somebody who was trained and then stopped, and say plainly that this is the person you most want to hear from. It is an uncomfortable request to make, and in our experience it is usually granted, because the account contact wants to know the answer too.