Vanity Metrics: Lipstick On A Pig
Many of the organizations fall into a trap of generating too much data (points) and reports as an alternative to good processes. Once you go down this path, there’s no dearth of exciting folks in the organization who start to measure every little thing happening around them. End result: everyone in the organization starts producing metrics to make them look good, even though the system as a whole is down. We call them Vanity Metrics – metrics that are simply meant to make you look good. Most of the times, they are not actionable (we all know why – what’s the point when everything looks so good!). They slowly bleed the organizations to death.
Well, what’s wrong in showing things in a good light? Nothing. As long as it is all aligned to your organization’s value stream and business goals. Typically, vanity metrics become a habit when end-to-end business transformation (read: system optimization) is not encouraged or thought of. As a result, everyone starts to think of their particular areas and ways to make them look good in a constraint environment (read: local optimization). Result: Good looking vanity metrics from each team; business continuous to suffer.
Switch over to Business Metrics – the actionable metrics aligned to your business and goals; metrics that show where your business is heading and what specific actions you need to take to keep closer to your goals. For product companies, a great source of metrics can be triggered through A/B Experiments. You don’t have to wait till production to figure out the customer impact. Incremental demos are good but they too serve the limited purpose.
A/B Experiments works great for UX centric changes. Next time you go for a BETA program, or internal validation – split your customers and provide them more than one way of doing the same thing (A and B). Gather the feedback on A and B ways of doing the same thing. It’s much valuable doing this while you are still developing your product than learning it through hard way in form of customer issues later on. Similarly, use Funnel Metrics and Cohort Analysis to serve you the best.
Key thing: define metrics that show the health of entire system (business/product) and serve the interest of all stakeholders. Stop focusing too much on metrics that are catering to particular teams or individuals. And be agile when it comes to metrics; continuously tune them to keep them aligned to your business goals.
Next time you think of Vanity Metrics, think of ‘lipstick on a pig’? It never works.