One Metric To Rule Them All!


People want focus! Move one metric! Up and to the right! Go non linear!

How do you focus an entire team or company? You just need to hold up One Metric To Rule Them All! I have heard this many times, and overheard a CEO talking about it just last week. I get it. Clarity is so much easier to rationalize and measure, so who wouldn’t rather have a black and white world?

The problem is that it is easy to go blind and start on a path that leads you to at best a local maxima, and most commonly to a rather bad spot, especially if you land on a vanity metric.

“What gets measured gets managed.” — Peter Drucker

While measuring is important, the key piece is making sure you are measuring the right things, and that you have a tension in your measurements. Take the time to have metrics that are in tension, or make sure that you put restrictions on the key metrics. Why?

Using the Levers

It is often easier than you think to force one metric. You often have multiple dials at your disposal, and you can point them at the metric at a detriment to the business. I learned this first hand in the world of eCommerce, where I was surprised to see how it was relatively easy to hit raw revenue numbers if you discounted profit. Your levers include pricing and buying traffic.

Vanity Metric Abuse

Input metrics can also be gamed. I recently saw someone who was measured on the number of people they were able to get to particular events. To juice the number, they were incented to get butts in seats with little regard for how qualified the people were. At some point they mentioned the fact that they even asked family to show up at times!

Quality vs. Quantity

To combat this, you have to balance quality vs. quantity. This is true in almost anything that you think of. Let’s take the number of people in prison. Surely we want less of them, right? The US has far too many people in the penal system, because it is ultimately unfair, as explained in this great book by Adam Benforado. We need to rethink policy and the incentive structures, and carefully balance the goal of having less people in prison, with the quality of people in there. In fact, the north star metric is a safe and fair society, and when you look at the problem through that lense, you broaden your view. For example, you remember that the system is meant to minimize offences, and isn’t there as a weapon of vengeance.

Ecosystem Views

This is why it is important to take an ecosystem view. I talked a little about this in my last post on the Web Ecosystem and how we view something as complex as the Web. The community needs to think of itself as regulators, looking to make changes that nudge things in the right direction. This broadens the view from “we make and improve our browser” (product) to “we do all we can to enable a healthy ecosystem, using all of the tools and investment options at our disposal”.

Short term vs. Long term

When you wear a regulator-type-hat, you are naturally thinking about the long term. This is important, as without this responsibility you are too tempted to get short term gains, which harm the long term. One of the root issues with climate change is the fact that manufacturing and the complex ecosystem didn’t have to factor the effects on the ecosystem into their market costs. This is what leads to the fact that a viable business can be made from flying water from Fiji around the world.

Add the fact that many workers have shorter stints in a role, or at a company, and you have to make sure that the incentive systems in place don’t force the short term. How does the performance review and promotion system think about work? Is your public company focused on the next quarters results for the market? Entropy is a strong force here to combat!

Segmentation and the long tail

It is important to spend time understanding the ecosystem you live in, and segmenting it correctly. Different ecosystems have different needs. The Web is naturally more long tail than other software ecosystems. The fact that everything is easily linked, mated with products such as Search that need to respond with an answer to many a niche query, force things in that direction. I personally love this about the Web, as it aligns the platform with allowing new incumbents and drives traffic to a greater variety of experiences.

Time to map it all out

So, take the time to really think through your ecosystem. First set north star output metrics for the ecosystem, then segment and learn what is happening, and finally have a strategy for what you want to change. From the strategy you can have the secondary metrics that can be leading indicators for the change that is coming (with ecosystems, often slower than you would like).

Make sure that you aren’t doing long term harm with your tactics. For example, with our love of “engagement”, it is easy to build viral systems that drive the wrong kind of engagement for our higher goals (not all engagement is equal). This is where your values come in (and often, strong founders) to make sure that you haven’t lost sight of the mission at hand.

This may be more work than driving ahead with that one metric to rule them all, but it’s worth it. And, there is even a “law” behind the danger, Goodhart’s Law, which Emily Freeman just brought top of mind:


How about building a model for the lifetime value of developers for your developer product? Ian Barber has a great analysis that he posted today!

Mission: Improve the Web Ecosystem for Developers

Once the dust settled on Google I/O 2018, it was time for me to find a new adventure. I wanted to share where I ended up because I am quite excited about it!

A few teams within Google have joined forces inside Chrome to focus on improving the Web ecosystem, focused on those who build experiences, and create on the Web. I am incredibly excited to be working side by side with Ben once again, and take on a new mission:

“We want to make high quality experiences easy to build as that will enable more meaningful engagement on the Web for users and developers alike.”

What is meaningful engagement?

We have to be careful what we measure. If we only cared about the amount of time that was spent on the Web, it would be tempting to do things that addict and actually cause harm….. just to get a number further up and to the right. We have all experienced sites and apps that seem to do just that, and it goes against our goals for humanity. Google wants to “organize the world’s information and make it universally accessible and useful.” Useful, not wasteful. We want to be a force for good.

Some of the core attributes that show the nuance of what we really want are:

  • We are not striving to get humans to spend more time on devices. This is a strong non-goal, and we care about digital wellbeing.
  • When someone spends time on a device, we would love their time on the Web to be with a great experience.
  • When someone wants to get a task done, we would love to streamline the effort which will result in less time spent on that activity, which is great!
  • We aren’t here to judge the type of activity. E.g. entertainment is meaningful for our users.
  • High performing experiences enable engagement because: Poor performing experiences result in abandonment, frustration, and platform switches; Each good experience results in myelination for reengagement
  • We want all users (reach) to be able to engage with the best possible experience (capabilities) for them.

There is a ton of nuance, and it is fun to spend time as a team thinking about what we want the world to look like, and drive for that reality.

How is the ecosystem doing?


While we have a fantastic, creative, community, it is clear that the Web that you experience daily does not map to the current capabilities of the Web. We are looking to work together to make changes that help close the gap, as when we see great modern experiencecs ship they see huge upside.

tl;dr Pinterest’s example here is the prototype turn around situation. Once a team focused on making their experience great it “became the top platform for new signups and in less than 6 months already have 800 thousand weekly users using our PWA like a native app from their homescreen”).

The future is already here — it’s just not very evenly distributed

One of the core issues at hand is shortening the time from “idea that makes the Web better” to “developers can use this idea and it is better for users”. There are a huge number of steps in between, and we need to shorten each of them. For example:

WEB PLATFORM LAYER

Web standards: Alex Russell recently posted two articles on the world of standards and lessons for becoming more effective. WICG and friends have both brought more web dev practitioners into the fold, and also streamlined the work. Getting consensus takes time, as it kinda should when adding something to the Web platform at this stage of its evolution, but that community is getting better at it over time.

Web compatibility: it is one thing to have a standard, it is another to have compatible implementations for developers. The Web Platform Tests effort has been huge here, and we see continued convergence across the browsers, which results in predictability for developers and users.

Web runtime: it is one thing to have implementations, it is another to have them be the implementations that are used. Chrome started the charge with rolling 6 week release cycles, and pushing continuous updates to users so they are running on the latest and greatest Web. Other browsers have also gotten better here. Being able to update here is critical. Time and time again we have seen situations where users are left on a device that isn’t getting updates and they are left in the past.

WEB APPLICATION LAYER

We see many pieces at play at the Web application layer:

Polyfills: One way to keep working towards the future is to be able to code to it’s interfaces even if all browsers in the wild aren’t there yet. This is where polyfills come in, as without them you either leave users out in the cold, or you are stuck back in time yourself. Getting your loading strategy working up front is important here. We still see, time and time again, web applications service polyfills to browsers that support the features natively. This eats up precious network and CPU, for no reason. It is important that a plan is in place for progressive enhancement that allows you to reach everyone, but deliver the best possible experience when capabilities match.

Frameworks: How do you keep up to date with the versions of frameworks? It can be tedious to keep upgrading, especially with large API changes. What is your strategy for keeping up to date here? For many, the “frameworks” are CMSes such as WordPress and Drupal. It was a huge win for the Web when WordPress supported updates inside of the admin panel. If you have a CMS that requires you to download zip files and do manual work to update…. you are probably spending more time in an unsecure past.

Libraries: We all know how those npm installs can blow up our dependency tree of libraries, so it is important to keep a watchful eye on these and once again have a plan for staying up to date. The nature of this work is tricky, and it is easy to be working with older versions of libraries based on transitive dependencies.

Your App Code: Oh right, you have to write the app itself! Hopefully you have A/B/release infrastructure in place that allows you to roll out the future to some users so you can see if that future is bright or not. With service workers, you have even more control of your application code, but that control means you need to make sure you have bulletproof infrastructure in place so you don’t lose users in the past as they get stuck in a historic cache.

CLOUD / OS LAYER

I remember the days where you had to keep your kernel up to date with the latest in security and performance patches. Fortunately, for many of us, those days are somewhat behind us, which is going to be important for future proofing.

The infrastructure that I would love to see in place, would allow for changes such as a new image format is introduced that is waaaaay smaller and faster:

  • Some browsers implement support and users get the capability in their hands
  • A WASM implementation is made available for browsers that don’t have native support
  • The browser and server are smart enough to negotiate and serve the new format in an optimal way.

Ditto for new networking solutions (as we rolled out QUIC et al). One of the fantastic things about AMP is how this world is available. A developer only needs to create AMP documents, and the rest is taken care of for you. And, with the AMP team constantly improving the runtime and the components, the same AMP page can keep getting better without you lifting a finger. With Web Packaging and the like, it would be great to do more of this outside of AMP.

One of the key notions of the progressive web, is that we are continuously improving. It is one thing to get the processes and platform chain of improvement working, and it is another to …

Make high quality experiences easy to build

You will notice “high quality” and “easy” are both in the sentence above.

Paul Calvano’s great piece on page weight

HIGH QUALITY

We harp on performance a lot, and the cost of JavaScript in particular, these days. We have taken a medium that was designed for streaming content and fast progressive loading and chopped it at the knees. Web development should offer you a pit of success but instead many stacks deliver a poor experience by default. If we could come together to fix one thing, it would be this.

It is one thing to run Lighthouse on an old legacy site and see a low score. You sigh, and know that you need to slog through a slew of work to bring it back to snuff. However, it is another thing to build a brand new app and run Lighthouse on it and …. also see a really poor score. The future we need starts you off high, and keeps nudging you to stay there. It has constraints and a mental model that allows you to scale your application. If you keep adding functionality, the first time to load can’t keep getting longer. At runtime, we can’t end up with a DOM with a million nodes, where the browser can’t keep up on layout, even if you are actually just making changes in one “view”.

Quality isn’t solely about performance though. We all want a Web that is also: capable, reliable, delightful, discoverable, and accessible.

I often think about this when I am on a website that offers the opposite. I had two such experiences in the last couple days:

Commerce experience: I was asked to purchase a gift card from a well known retailer. I first tried to grab one quick on a mobile device. The UI kept jumping around and the form controls actively stopped me from doing correct selections. I then jumped to the desktop version, and it was equally bad, giving me bizarre error messages that made no sense (and were different to those that surfaced on mobile). I then turned to the mobile application to make the purchase and pictured the data folks looking at these types of experiences and thinking “ah, so users of the app have better conversion!” No!

Push the app experience: I was looking to answer a question and ended up on a website that did everything possible to get me to download the app. There are plenty of reasons that I may want an app on my phone, but the “frustrate the user to do so” drives me insane. I buggered off to the next search result to get what I needed. I also picture data folks there looking at engagement and doing comparisons that don’t take into account that they are measuring different user segments. I have run into this time and time again, where the app users are those who waded through everything to get the app and are thus the loyal customers who put up with it. However, how about also opening the funnel to convert “not loyal yet” customers to the next level over time.

EASY

It’s too hard to build a great web experience. I made my own case here when you read through the infrastructure that you need to setup to build an engine that delivers up to date experiences. We have a paradox of choice, and it is easy to second guess yourself all day long (React or Vue? Web Components? Redux or MobX? Webpack or Parcel? etc etc etc).

Once you have made your choices, you have to make sure that they deliver a flow that is productive, and also delivers solid experiences (the PWA starter kit is an example of putting together the pieces that will keep you in the lane). The Web has plenty of footguns, so your experience can quickly become janky. To help us keep on the path we are improving the platform and also setting up Feature Policies that aim to guide. If you are able to policy up, the browser can optimize and users will benefit.

With all this being said, it is easy to miss out on the fact that the Web is still the best way to build experiences that reach users across form factors, and it has key principles that can’t be beat. Being able to fully control deployment on one end, and be able to make live changes in development on the other, is an amazing combination. If we can push through and have a nice native setup that allows for the best of these worlds without making developers live in config hell, we will be in a great place, and with recent changes we are able to get there.

We need a power assistant to help make this all happen. We have so many of the pieces already:

  • Lighthouse as a way to continuously understand how your development, staging, and production are looking in the lab
  • Chrome User Experience Report, Page Speed Insights, and friends to get you data from the field
  • Chrome DevTools is there to help you inspect, develop, and debug your applications
  • There are a few great editors, that are able to tie into the ecosystem
  • Community efforts to guide you down the path.

But we need more. We need to give developers better guidance and help them along. We need the tools to give more insights and give you greater help. You should be able to do more development and design work in DevTools. We need to help frameworks and ecosystems such as CMSes (e.g. WordPress PWA) and Commerce deliver great experiences to a huge part of the Web.


Guess what? We are hiring. If you would like to help us take everything to the next level and light the way to the pit of success, please let me know! We have positions across Product Management, Engineering, DevRel, and more. If you want to help the ecosystem and platform, we are all ears. Please join me on the new journey, one that will help the developers of today, and help us get to the future of a Web across new modes and form factors.

Google I/O 2018: Integration in the Open


With Google I/O 2018 kicking into gear, I found myself looking back at my thoughts on last years event and thinking about what is changing and what is continuing with respect to the journey for our developer platforms and ecosystems.

Technology exists within the framework of what is going on in the world. On the one hand, the world gets better for more people every day, but on the other hand we also know how fragile things are, and how much pain people are in as we see growing inequality in areas. Within this context we are seeing technology companies look deeper into themselves, thinking more about what is being created. Are we building experiences that are meaningful for people? That is the question at hand, and the consumer keynote shows some of what Google is thinking about here.

The Integration Story

One of the foundations of integration is componentization, and we are seeing great strides here across our platforms.

Android’s new Jetpack libraries accelerate development

This year builds on last years Kotlin news, and shows our vision for mobile Android development. App bundling and dynamic delivery allows you to split up your app, giving users the ability to download the pieces they need. If you want to reach all of your potential users and deliver the best possible experience to them, you should be focusing on the vitals, and our tooling and platform is working on making life much easier for you here. With Actions and Slices, you can bring pieces of your app to new surfaces on Android to surface where users want them.

Android Studio 3.3 brings a slew of improvements and cuts a huge amount of time out of your development. Did you see Tor demo how quickly the emulator starts now? With Jetpack, we bring together the best of architecture components with the support library, allowing you to develop much faster, reach more users with the results, and all with top notch Kotlin support.

And this year Android really hits new form factors hard with great updates across Wear, TV, and Android Things hitting 1.0.

Spotify shows off their Desktop PWA

On Web, if you follow the Lighthouse (newly updated for I/O!) well-lit path, you will be able to deliver ephemeral experiences that integrate deeply with the host operating system. Performance is still the number one area of focus, but you will see that we are broadly looking to get rid of friction wherever it lay. Web packaging takes fast and makes it instant, and it is now simple to get your users signed up, signed in, and able to quickly pay.

While the first PWAs that drew attention were mobile, with desktop PWA support we get to see how your investment scales. Twitter has been able to take their “lite” mobile app and bring it to the desktop/tablet, Windows store, and with the Chrome desktop PWA affordances you get an amazing experience that marries the your own focused app with browser features such as URL sharing, find in page, casting, etc. This has allowed Spotify to integrate their web experience, for Starbucks to double the usage of their PWA.

Chrome DevTools continue to improve with end to end support for the latest in layout (grid), JavaScript (eager eval, async/await, etc), and even end to end debugging for your node services. With Lighthouse baked in and lighting the way, you can see how far the frameworks are coming along to help deliver the fast experiences the Web needs. Polymer 3 and lit-html allow you to build blazingly fast experiences that are close to the metal, and integrate so well with Redux and friends. Angular 6 joins the custom elements party and the Ivy runtime shows how much of a diet it has been on for great enterprise products. And AMP is here for an opinionated fast path with support for Stories and other new components that you can drop in and use. The Web is here for you, from rich content to complex apps that can tie into new ecosystems with WebAssembly.

Turn on the new Linux support in ChromeOS settings

You may notice more Pixelbooks at I/O this year. I have been using desktop PWAs a lot recently, as I have been living on a Pixelbook. I had tried to make the switch to ChromeOS several times, and this is the first time that it has stuck.

I used to feel constrained inside a browser, but this has flipped since I changed the way I used the system. On my desktop, I normally have a set of pin’d tabs and Ctrl/Command-# to jump to a particular main app (email, calendar, tasks, chat). On ChromeOS I pin those as apps and Alt+# to jump to them.

This frees up my browser. The “email” browser turns out to be my main source of tabs, and I see a new pattern forming. It turns out that a lot of “apps” are sources for browser tabs. I click on links in Gmail AND from Calendar AND Asana AND Chat AND Twitter. This drives an interesting world where the first tab is the main source of other experiences that can be tied to that app instance.

Having the desktop UI (three vertical dots) available brings the best of the browser to the app. The user agent is out of the way, but you have a quick path to find in page, font sizing, casting, autofill, and copying the URL for easy sharing…. and you can imagine so much more.

With the release of secure Linux apps via VM/container goodness, I am able to run my favorite developer tools (Visual Studio Code, Atom, Android Studio) as well as other key Linux apps (including other browsers for testing). It’s still early, but it’s already great to be able to develop here.

Linux and the Web is great, but what about access to the entire Android and Assistant ecosystem? It’s all here, and with the world class security of ChromeOS (and ability to login to a new machine and be right where you left off… which feels like my college days with XWindows!).

Reach the assistant through Web and Android

In fact, the Assistant, and Actions on Google in general, is another perfect example of integration. Web developers, Android developers, and backend developers should all be able to get their experiences in front of Assistant users. With Actions for these platforms, you can get a broad reach, from something quick and direct all the way to a full conversational experience across text and voice, and with RCS you can leverage the texting experience all the way to Android Auto.

ARCore: now with Sceneform, Augmented Images, and Cloud Anchors

AR and VR is yet another example, where the technology is available for you to use the world around you. I can’t wait for a future browser for the world (whether it be Earth or something else immersive), and that world is a lot closer with the new tools that we announced today:

  • Sceneform is a new 3D SDK for Java developers
  • Augmented Images allows you to associate AR content with the real world
  • Cloud Anchors bring a shared collaborative environment to life. Your apps, on iOS or Android, can manipulate space in a way that all of your users can see.

ML SDK across the platforms; from edge to cloud

The other piece of integration is the role of client and server. We continue to live in a disconnected world, with variable levels of compute and network power on our devices married to amazing levels of compute on the server. Developers need the ability to drive as much as possible on the edge where it makes sense, but also use the power of the data center where needed. A strong theme at I/O continues to be the role of machine learning and AI, and not only do we have a lot to share on the side of Google Cloud and TensorFlow, but we keep bringing that to the client with APIs such as ML Kit, able to reach all platforms via Firebase.

In fact, ML Kit is able to infer on device, but also use the Cloud to get more detail (e.g. maybe on device detects a “bridge” in a photo, but the Cloud comes back with a more detailed “Golden Gate Bridge”).

The APIs in the initial kit include:

  • Text recognition
  • Image labeling
  • Barcode scanning
  • Face detection
  • Landmark recognition
  • Smart reply

And you can expect a lot more to come!

Same great design system, themed your way

The last piece is the UI, and we have a major upgrade to the world of Material Design this year. People have noticed our own properties get large upgrades, such as the recent Gmail redesign, and this work is on the back of the new Material Theming approach. Companies have wanted to take the material design system, and really brand it to their needs, and with the new theming system it is much much easier to do just that.

The new system comes with new tools, such as the Theme Editor, which are one of a series of updates where we try to help you build rich experiences with more ease. All of the components are released as open source, for Android, iOS, Web, and now also for Flutter. Flutter announced beta 3 which brings large improvements, and could be a perfect fit for your cross platform needs.

Phew, this is only the tip of the iceberg on what we have in store for I/O this week. The teams have been working hard to build useful things, and share them throughout the week with great talks. We will be rushing to get the content up as soon we can after the livestream.

One last thing. Watching and listening in is great, but there is nothing like touching the technology and seeing where things really are. This is why I always dart over to the codelabs area and try everything out.

Google I/O is truly about the integration of everything that we have to offer across our ecosystems. In that spirit, it’s most fun to see how these things work together and build bridges.

https://mobile.twitter.com/kelseyhightower/status/993846267976470528

I have always been drawn to open ecosystems, so I love that openness is baked into our DNA. I love looking at the history of Android, Chromium, TensorFlow, Kubernetes, Flutter, Polymer, Angular, AMP, gVisor, and the myriad of over initiatives that have open source at the core.


“platforms integrate
together in the open
all for us humans” — Steven Colbert #io18