Photo by David Birozy on Unsplash
QA is a Hidden Lever to Increase Profit
Software testing reduces churn, and less churn leads to more profit
Let's get straight to the good stuff - reducing churn can probably help your business retain hundreds or even thousands of dollars in customer lifetime value. QA helps to reduce the problems that contribute to churn, thus helping to increase profit.
Small changes to churn can have large effects on profitability. In financial services, a 5% increase in customer retention can improve profits by 25%. In web services, a company with a negative 5% net churn can be 73% more profitable than a company with a 5% churn rate! So how can we reduce churn? QA can help!
Net Promoter Score: the pulse of churn
Ever go shopping and receive an offer to complete a survey at the bottom of your receipt? The information from this survey is often used to calculate a metric called Net Promoter Score (NPS), which essentially asks:
“How likely is it that you would recommend [our product, service, or company] to a friend or colleague?”
The goal of NPS is to quickly gauge customer loyalty, similar to how a person's pulse provides a quick gauge of health. If, as management guru Peter Drucker once said "the true purpose of a business is to create and keep customers," then NPS measures how many customers you are keeping versus how many customers you are losing.
NPS for your product is based on the percentage of customers who are:
promoters
- the enthusiastic customers of your product
passives
- satisfied, yet unenthusiastic customers
detractors
- unhappy customers who can impact growth through negative word-of-mouth
For example, if 70% of your customers are promoters, 20% are passives, and 10% are detractors, your NPS will be 60:
NPS calculation:
Knowing your product's Net Promoter Score and that of your competitors gives your QA team a starting point to communicate with other company stakeholders about the importance of QA. By helping to find bugs, prevent defects, improve performance, and create an exceptional user experience, QAs can help a product gain more promoters and fewer detractors.
Want more sales? Fix churn.
Advertising legend Bill Bernbach left an indelible imprint on the minds of Americans in the 1980s. If you have ever heard about Juan Valdez harvesting Colombian coffee or that Mikey likes Life Cereal, you can thank Bill Bernbach. Needless to say, he knew how to market a product. But as he would tell you:
"A great ad campaign will make a bad product fail faster. It will get more people to know it's bad."
In other words, if you're going to market a product, it should be a good product first! Spending more on Google Ads or hiring a superstar sales team will only make problems worse if you don't fix your product. Thinking back to NPS, in the long run, attracting attention to a buggy product will only succeed in creating more detractors and digging your product into a deeper hole.
How many defects does it take for a customer to churn? Unfortunately, not many. (Mr. Owl would say "three.") QA industry research suggests more than 80% of people may abandon an app if it has enough bugs, so even a few persistent bugs will likely start to cause churn. By advocating for resources to find and fix bugs, QAs can address some of the root causes of churn.
What software problems cause churn?
In the paper "On the Relationship between User Churn and Software Issues," academic researchers analyzed comments from the website alternativeto.net to look for predictors of user churn. They identified three main software-related causes:
"Issue Time Alive" - the longer a defect remains in production, the more upset customers become
"Number of Comments" - if more customers are affected by a bug, the more likely there will be at least one upset customer
"Time since last change" - code that changes frequently ("code churn") is much more likely to contain bugs
If you want to do a similar deep-dive for your app, websites such as g2.com and Capterra provide alternatives and user reviews for most business software. For mobile apps, you can read user comments from the App Store or Google Play.
What QAs can do to help reduce churn
Frequently test the parts of the app customers use most
A bug in a part of your app that many customers use is more likely to upset users than a bug in an obscure corner of the app. Focus your efforts on testing the features that matter most, and make sure your product team prioritizes these tests even if those features aren't in the current sprint.
End-to-end automated tests provide the most value when they are run frequently on workflows customers use most. Repeated testing should help prevent bugs from arising the same way in these areas - in fact, the Pesticide Paradox is predicated on this. Ironically, if your automated regression tests for the most common workflows don't find any bugs, that is the outcome you want!
Frequently test parts of the app subject to code churn
Code created or changed recently is more likely to contain bugs. Code that is rewritten repeatedly is even more likely to contain bugs, as evidently, the last code change didn't solve the problem! Identify sections of your app's codebase that have recently or frequently been rewritten and figure out which app features they affect. During exploratory testing, the app features affected by recent or frequent code changes should be among your usual suspects to find potential bugs.
Perform visual tests on frequently viewed content
While end-to-end automated tests tend to focus on functional testing, most test automation tools do not view Web and mobile app content the same way users do. As I pointed out in a previous article, if there is a bug related to the visual appearance of an app, a functional test may not detect it.
Here is an example of a visual bug I found on the website of a large US real estate developer. Would you feel completely comfortable logging into this page?
Always consider your product from the perspective of a user, and keep in mind that most users of your app do not have a great deal of background knowledge in software development. When a user sees something going wrong in the UI, they will infer that other things are also going wrong. A highly polished UI, however, will reassure users and give them greater confidence in the overall quality of your product.
Performing visual tests manually can work for sections of the app that users don't visit frequently, but for your most important pages, an automated solution like Applitools will help you test more frequently, thus finding unsightly visual bugs before they undermine the confidence users have in your app.
Improve product performance
Not all problems are functional. An app may function correctly, but if the app appears slow and unresponsive, the user experience will suffer.
Performance testing is not always part of the everyday stream of software testing work, but no less important. In a typical Agile team, developers, testers, and product managers may schedule a sprint and work on stories together to gradually assemble product features. Once a product feature is released to production, the developer team may not work on the feature every day, but DevOps certainly will as delivery of the app and its data take on a greater role.
Testing and improving the performance of a system, particularly in an ongoing automated manner, is essential to creating an outstanding user experience. You don't just want to satisfy customers - you want to delight them! Enthusiastic customers will become the promoters that drive your Net Promoter Score, and providing those customers with a performant app is an important part of that process.
QA can unite stakeholders to reduce churn
Since QAs often test the outputs of other team members, people sometimes lose sight of the greater contributions that QA can make. By uniting company stakeholders such as finance, sales, operations, and engineering around metrics like Net Promoter Score, QA can help create the conditions to improve customer satisfaction and facilitate sales.
Reducing churn is a company-wide effort; by providing the information to identify and solve engineering problems leading to churn, QA is well-positioned to coordinate efforts across the company to reduce churn, thus improving profitability. And what company doesn't like making more money?