People want focus! Move one metric! Up and to the right! Go non linear!
How do you focus an entire team or company? You just need to hold up One Metric To Rule Them All! I have heard this many times, and overheard a CEO talking about it just last week. I get it. Clarity is so much easier to rationalize and measure, so who wouldn’t rather have a black and white world?
The problem is that it is easy to go blind and start on a path that leads you to at best a local maxima, and most commonly to a rather bad spot, especially if you land on a vanity metric.
“What gets measured gets managed.” — Peter Drucker
While measuring is important, the key piece is making sure you are measuring the right things, and that you have a tension in your measurements. Take the time to have metrics that are in tension, or make sure that you put restrictions on the key metrics. Why?
Using the Levers
It is often easier than you think to force one metric. You often have multiple dials at your disposal, and you can point them at the metric at a detriment to the business. I learned this first hand in the world of eCommerce, where I was surprised to see how it was relatively easy to hit raw revenue numbers if you discounted profit. Your levers include pricing and buying traffic.
Vanity Metric Abuse
Input metrics can also be gamed. I recently saw someone who was measured on the number of people they were able to get to particular events. To juice the number, they were incented to get butts in seats with little regard for how qualified the people were. At some point they mentioned the fact that they even asked family to show up at times!
Quality vs. Quantity
To combat this, you have to balance quality vs. quantity. This is true in almost anything that you think of. Let’s take the number of people in prison. Surely we want less of them, right? The US has far too many people in the penal system, because it is ultimately unfair, as explained in this great book by Adam Benforado. We need to rethink policy and the incentive structures, and carefully balance the goal of having less people in prison, with the quality of people in there. In fact, the north star metric is a safe and fair society, and when you look at the problem through that lense, you broaden your view. For example, you remember that the system is meant to minimize offences, and isn’t there as a weapon of vengeance.
Ecosystem Views
This is why it is important to take an ecosystem view. I talked a little about this in my last post on the Web Ecosystem and how we view something as complex as the Web. The community needs to think of itself as regulators, looking to make changes that nudge things in the right direction. This broadens the view from “we make and improve our browser” (product) to “we do all we can to enable a healthy ecosystem, using all of the tools and investment options at our disposal”.
Short term vs. Long term
When you wear a regulator-type-hat, you are naturally thinking about the long term. This is important, as without this responsibility you are too tempted to get short term gains, which harm the long term. One of the root issues with climate change is the fact that manufacturing and the complex ecosystem didn’t have to factor the effects on the ecosystem into their market costs. This is what leads to the fact that a viable business can be made from flying water from Fiji around the world.
Add the fact that many workers have shorter stints in a role, or at a company, and you have to make sure that the incentive systems in place don’t force the short term. How does the performance review and promotion system think about work? Is your public company focused on the next quarters results for the market? Entropy is a strong force here to combat!
Segmentation and the long tail
It is important to spend time understanding the ecosystem you live in, and segmenting it correctly. Different ecosystems have different needs. The Web is naturally more long tail than other software ecosystems. The fact that everything is easily linked, mated with products such as Search that need to respond with an answer to many a niche query, force things in that direction. I personally love this about the Web, as it aligns the platform with allowing new incumbents and drives traffic to a greater variety of experiences.
Time to map it all out
So, take the time to really think through your ecosystem. First set north star output metrics for the ecosystem, then segment and learn what is happening, and finally have a strategy for what you want to change. From the strategy you can have the secondary metrics that can be leading indicators for the change that is coming (with ecosystems, often slower than you would like).
Make sure that you aren’t doing long term harm with your tactics. For example, with our love of “engagement”, it is easy to build viral systems that drive the wrong kind of engagement for our higher goals (not all engagement is equal). This is where your values come in (and often, strong founders) to make sure that you haven’t lost sight of the mission at hand.
This may be more work than driving ahead with that one metric to rule them all, but it’s worth it. And, there is even a “law” behind the danger, Goodhart’s Law, which Emily Freeman just brought top of mind:
How about building a model for the lifetime value of developers for your developer product? Ian Barber has a great analysis that he posted today!