• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Dion Almaer

Software, Development, Products

  • @dalmaer
  • LinkedIn
  • Medium
  • RSS
  • Show Search
Hide Search

Metrics

One Metric To Rule Them All!

August 11, 2018 Leave a Comment


People want focus! Move one metric! Up and to the right! Go non linear!

How do you focus an entire team or company? You just need to hold up One Metric To Rule Them All! I have heard this many times, and overheard a CEO talking about it just last week. I get it. Clarity is so much easier to rationalize and measure, so who wouldn’t rather have a black and white world?

The problem is that it is easy to go blind and start on a path that leads you to at best a local maxima, and most commonly to a rather bad spot, especially if you land on a vanity metric.

“What gets measured gets managed.” — Peter Drucker

While measuring is important, the key piece is making sure you are measuring the right things, and that you have a tension in your measurements. Take the time to have metrics that are in tension, or make sure that you put restrictions on the key metrics. Why?

Using the Levers

It is often easier than you think to force one metric. You often have multiple dials at your disposal, and you can point them at the metric at a detriment to the business. I learned this first hand in the world of eCommerce, where I was surprised to see how it was relatively easy to hit raw revenue numbers if you discounted profit. Your levers include pricing and buying traffic.

Vanity Metric Abuse

Input metrics can also be gamed. I recently saw someone who was measured on the number of people they were able to get to particular events. To juice the number, they were incented to get butts in seats with little regard for how qualified the people were. At some point they mentioned the fact that they even asked family to show up at times!

Quality vs. Quantity

To combat this, you have to balance quality vs. quantity. This is true in almost anything that you think of. Let’s take the number of people in prison. Surely we want less of them, right? The US has far too many people in the penal system, because it is ultimately unfair, as explained in this great book by Adam Benforado. We need to rethink policy and the incentive structures, and carefully balance the goal of having less people in prison, with the quality of people in there. In fact, the north star metric is a safe and fair society, and when you look at the problem through that lense, you broaden your view. For example, you remember that the system is meant to minimize offences, and isn’t there as a weapon of vengeance.

Ecosystem Views

This is why it is important to take an ecosystem view. I talked a little about this in my last post on the Web Ecosystem and how we view something as complex as the Web. The community needs to think of itself as regulators, looking to make changes that nudge things in the right direction. This broadens the view from “we make and improve our browser” (product) to “we do all we can to enable a healthy ecosystem, using all of the tools and investment options at our disposal”.

Short term vs. Long term

When you wear a regulator-type-hat, you are naturally thinking about the long term. This is important, as without this responsibility you are too tempted to get short term gains, which harm the long term. One of the root issues with climate change is the fact that manufacturing and the complex ecosystem didn’t have to factor the effects on the ecosystem into their market costs. This is what leads to the fact that a viable business can be made from flying water from Fiji around the world.

Add the fact that many workers have shorter stints in a role, or at a company, and you have to make sure that the incentive systems in place don’t force the short term. How does the performance review and promotion system think about work? Is your public company focused on the next quarters results for the market? Entropy is a strong force here to combat!

Segmentation and the long tail

It is important to spend time understanding the ecosystem you live in, and segmenting it correctly. Different ecosystems have different needs. The Web is naturally more long tail than other software ecosystems. The fact that everything is easily linked, mated with products such as Search that need to respond with an answer to many a niche query, force things in that direction. I personally love this about the Web, as it aligns the platform with allowing new incumbents and drives traffic to a greater variety of experiences.

Time to map it all out

So, take the time to really think through your ecosystem. First set north star output metrics for the ecosystem, then segment and learn what is happening, and finally have a strategy for what you want to change. From the strategy you can have the secondary metrics that can be leading indicators for the change that is coming (with ecosystems, often slower than you would like).

Make sure that you aren’t doing long term harm with your tactics. For example, with our love of “engagement”, it is easy to build viral systems that drive the wrong kind of engagement for our higher goals (not all engagement is equal). This is where your values come in (and often, strong founders) to make sure that you haven’t lost sight of the mission at hand.

This may be more work than driving ahead with that one metric to rule them all, but it’s worth it. And, there is even a “law” behind the danger, Goodhart’s Law, which Emily Freeman just brought top of mind:

I've had a few chats this week about measuring #devrel efficacy. I found a term for the thing we all talk about! pic.twitter.com/dpWlJR8HRL

— Emily Freeman (@editingemily) August 12, 2018


How about building a model for the lifetime value of developers for your developer product? Ian Barber has a great analysis that he posted today!

Primary Sidebar

Twitter

My Tweets

Recent Posts

  • I have scissors all over my house
  • GenAI: Lessons working with LLMs
  • Generative AI: It’s Time to Get Into First Gear
  • Developer Docs + GenAI = ❤️
  • We keep confusing efficacy for effectiveness

Follow

  • LinkedIn
  • Medium
  • RSS
  • Twitter

Tags

3d Touch 2016 Active Recall Adaptive Design Agile Amazon Echo Android Android Development Apple Application Apps Artificial Intelligence Autocorrect blog Bots Brain Calendar Career Advice Cloud Computing Coding Cognitive Bias Commerce Communication Companies Conference Consciousness Cooking Cricket Cross Platform Deadline Delivery Design Desktop Developer Advocacy Developer Experience Developer Platform Developer Productivity Developer Relations Developers Developer Tools Development Distributed Teams Documentation DX Ecosystem Education Energy Engineering Engineering Mangement Entrepreneurship Exercise Family Fitness Founders Future GenAI Gender Equality Google Google Developer Google IO Habits Health HR Integrations JavaScript Jobs Jquery Kids Stories Kotlin Language Leadership Learning Lottery Machine Learning Management Messaging Metrics Micro Learning Microservices Microsoft Mobile Mobile App Development Mobile Apps Mobile Web Moving On NPM Open Source Organization Organization Design Pair Programming Paren Parenting Path Performance Platform Platform Thinking Politics Product Design Product Development Productivity Product Management Product Metrics Programming Progress Progressive Enhancement Progressive Web App Project Management Psychology Push Notifications pwa QA Rails React Reactive Remix Remote Working Resilience Ruby on Rails Screentime Self Improvement Service Worker Sharing Economy Shipping Shopify Short Story Silicon Valley Slack Software Software Development Spaced Repetition Speaking Startup Steve Jobs Study Teaching Team Building Tech Tech Ecosystems Technical Writing Technology Tools Transportation TV Series Twitter Typescript Uber UI Unknown User Experience User Testing UX vitals Voice Walmart Web Web Components Web Development Web Extensions Web Frameworks Web Performance Web Platform WWDC Yarn

Subscribe via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Archives

  • February 2023
  • January 2023
  • September 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • November 2021
  • August 2021
  • July 2021
  • February 2021
  • January 2021
  • May 2020
  • April 2020
  • October 2019
  • August 2019
  • July 2019
  • June 2019
  • April 2019
  • March 2019
  • January 2019
  • October 2018
  • August 2018
  • July 2018
  • May 2018
  • February 2018
  • December 2017
  • November 2017
  • September 2017
  • August 2017
  • July 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012

Search

Subscribe

RSS feed RSS - Posts

The right thing to do, is the right thing to do.

The right thing to do, is the right thing to do.

Dion Almaer

Copyright © 2023 · Log in

 

Loading Comments...