How to decide which user feedback matters

User feedback is widely understood to be a cornerstone of product development, but not every user request should be taken literally.

Phoebe and I joined UQ Ventures' ilab Accelerator back in November with our startup Go Locum - the medical workforce marketplace that connects hospitals to doctors to fill urgent roster vacancies.

One of the main benefits of being in a program like this is the chemistry that arises when you get like-minded founders in the same room. There's a contagious optimism in the air that's hard to find elsewhere. There's also a lot to learn from founders facing similar challenges.

This article is inspired by a number of things I've learned from chatting to founders in the UQ Ventures ecosystem.

Which user feedback matters?

It's widely recognised in product development circles that user feedback is the most direct way to ensure that your product is heading in the right direction. For those familiar with the Lean Startup methodology, you may recall the Build-Measure-Learn loop:

The critical idea here is that companies that can:

1. make changes to their product
2. measure user feedback
3. learn lessons from that feedback

in the fastest way possible will ultimately succeed in the long run.

Putting systems in place to shorten the time it takes to get through a complete build-measure-learn cycle is considered a priority in the startup world, and in modern business thinking more generally.

For example, one tip to increase the rate of spontaneous user feedback is to make it as easy as possible for users to talk to you! I often see apps with the "Give Feedback" button very deep in the settings workflow, sometimes 2 or 3 clicks deep.

When I moved the "Send Feedback" button from the settings menu to front-and-centre of the Olog home screen, my user feedback queries tripled:

You should think broadly about what consitutes user feedback

User feedback is not just what people say to you about your product or app - it's also how they act. Especially if they're your family and friends!

A classic trap in the software world is to rely on family and friends to provide feedback to drive development - this is a huge problem, mostly because family and friends will almost exclusively make positive and encouraging marks, believing that they're doing the right thing in supporting you to follow your dreams.

This sounds great until you realise that this can send you down rabbit holes for months or even years with concepts that will never survive in the free market.

So, if we want to be careful analysing what people say and instead focus on what they do, what user actions should we care about?

Here are my thoughts. Do users:

1. Come back to your application multiple times?
2. Integrate your application into their workflow as intended?
3. Tell their friends about your project?
4. Complain about shortcomings in your product?

#4 might sound like a weird one to have on there, but I believe that negative feedback, especially lots of negative feedback, can actually mean that you're excruciatingly close to Product-Market fit. There's nothing more annoying than a tool that almost solves a really painful problem.

If you're getting very little to no feedback, or only very superficial positive feedback, it likely means users simply do not care about your app, and this is probably the worst outcome of all.

My own experience with these ideas as a developer and product manager has taught me however that not all feedback is equal. You can fall into a trap where a subset of your users report problems that really shouldn't be a priority for you to fix.

I thought I'd share some real examples of user feedback queries that I have received from my apps, and how I rationalised the actions I took in response to them.

Unravelling the meaning behind user feedback

Olog is a mobile app I built that helps doctors at Queensland Health digitally manage their overtime claims.

Here are a few feedback messages I've received from my users, and my associated thoughts.

#1

This is a feature request from a user who had lost the overtime paperwork generated by Olog. In this case, the feature itself might take half a day or so to build (once you include testing and deploying the new versions), however fixing this user's acute issue manually would only take about 2 minutes.

Once upon I time I would have opened my project immediately to try and fix this bug in the hopes of appeasing this user as quickly as possible. Nowadays though, I view these feature requests in a different way.

Hugh is really saying "I am having this problem with your app where I have lost my overtime form.".

This is a one-time issue, and so until many users start having this problem, spending the half a day to patch the app itself isn't worth it. For problems like this, I look for ways to manually (and quickly) solve their pain, if at all possible. That is the goal at the end of the day, and you might as well achieve it in the most time-efficient way possible.

#2

Here's another example of a feature request which is really a user complaining of an acute problem they want resolved. They've made a mistake in one of the forms they've submitted, and want to be able to go back and reverse this claim.

My options here would be to a) reverse the claim manually in the production database or b) patch the app to enable this functionality over the long term.

Right now this user only really cares about reversing the incorrect overtime claim they've filed, and likely won't care about this feature in the future after this claim is resolved.

So until this repeatedly becomes a problem, I will be manually undoing these claims for users. You get the same degree of user satisfaction for 1/100th of the time investment.

#3

So far we've seen a few examples of feature requests that are really the symptom of an acute problem that can be readily solved manually instead.

But what sort of feedback should be actioned with application fixes readily?

In example #3 we have received a message from a new user who can't work out how to use Olog properly.

This is quite an important message to respond to, given that the first impression a user has of your application is an opportunity to demonstrate your app's magic that you'll probably only get one shot at. The vast majority of app downloads will result in a single app session only.

In the book Product Led Growth, Wes Bush stresses the importance of users having a positive first experience with your product offering. The sooner you can get the user to their "happy moment", the faster your app will grow.

So when users report problems learning how to use your app, you should solve this as a matter of priority. Note that in this case, if you implemented this feedabck, you'd get substantial benefit for all future new users who join your app, not just a small subset of users who have a niche acute problem as in the above examples.

A user feedback analysis framework

So now when I'm analysing user feedback, I consider the following questions to help me decide how to respond to them:

• How complex would this feedback be to implement? If it's only a few minutes worth of work, it's likely worth stopping your analysis here and just implementing it. Scrutinise the feedback more heavily as the complexity increases.
• Is this feedback from a stranger, or a family member or friend? Strangers, especially those who are paying for your service, will provide the most honest and highest priority feedback.
• What proportion of my current or potential future users will benefit from implementing this feedback? Prioritise feedback that will benefit the highest proportion of users (or alternatively, the most strategically important subsets of users).
• Is this feedback the symptom of an acute problem I can fix manually? Do users encounter this problem frequently enough to warrant a long-term fix at this stage?
• Is this feature in keeping with the long term vision for the product? Or is it too specifically weighted towards one user's needs?

Have you fallen into a trap listening to the wrong user feedback? Do you have anything you'd add to this framework? I'd love to hear about it.