Good product leaders need to be ready to make tough calls, one of them is to kill feature(s) after a careful analysis.
This is a true history from my experience as a product leader when I have to kill a guest-checkout option in an e-commerce website to contribute to the overall business strategy. What about the customer? Well, after doing my homework – in other words: user research, competitor analysis, test prototypes, and running a/b testing-. The conclusion was that we also added long-term value to the customer.
When should we kill a feature?
- The feature is not being used: If a feature is not being used by a significant number of customers, it may not be worth the cost of maintaining and supporting it.
- The feature is causing more problems than it solves: If a feature is causing significant issues, such as bugs or user confusion, it may be time to remove it to improve the overall user experience.
- The feature no longer fits the product’s vision or goals: If a feature no longer aligns with the product’s strategic direction, it may be necessary to remove it to maintain focus.
- The feature is too expensive to maintain: If a feature is too costly to maintain, it may not be worth keeping it in the product.
- The feature is outdated: If a feature is no longer relevant or useful due to changes in technology or user behavior, it may be time to remove it.
In the case of the e-commerce, the guest-check out option did not longer fits the direction of product strategy and it was causing more problem than it solves.
A/B testing to kill a Feature
We did not immediately pull the plug on it. I know that removing the guest-checkout option will affect our conversion rate; in the first place, it was the reason it was created, so customers could finalize faster and succesfully their purchase journey without any blocker.
A good a/b test to kill a feature should answer the 3 next questions:
- Are there significant complaints being made by users?
- Will there be significant changes in user behavior indicating users are getting less value from the overall product?
- Are there changes in business metrics indicating less ability to capture value from users?
Similar to A/B testing for understanding the value of a new feature, Launching A/B testing to kill feature follow the same steps:
- Clearly define your goal
- Identify the metrics to measure
- Create two versions of the product: Two version are as similar as possible, except for the presence or abasence of the feature.
- Randomly assign users to each group
- Run the test for a sufficient period
- Analyze the results
Who makes the decision?
The decision to kill a feature is not for the product manager or only the product management department. Typically involves multiple departments within an organization. For instance: Customer Support, Marketing and Sales, Engineering, User experience, and others.
Ultimately, the decision to kill a feature requires collaboration and input from multiple departments to ensure that it aligns with the company’s overall goals and objectives.