Archives for March 2016

Remembering Simpler Times

March 29, 2016 Leave a Comment

Take a deep look, and you realize how backwards we were.

Kevin is sitting on a park bench. A cup of black coffee sits in one hand, and he tosses some bread out with the other, slowly watching the birds eat. If you were walking by you would have an odd feeling that Kevin has spent a lot of time there. Then you notice that there is space for another, with a subtle dip in the wood.

Then Brandon comes along, e-cigarette hanging in his mouth, and he joins Kevin. They sit there for some time before talking.

Reload The World

Kevin kicks things off. He talks about a time in the past, a time where bugs were easily hidden by the fact that an entire sandbox runtime was rebooted constantly. If that reloading didn’t quite work, the user knew to do an even bigger reboot. This was the time of the Web, where each click was a reload, and (Shift) Reload made sure things could come back. It was a world that felt janky and wrong. Why reload and repaint everything when so much of the page actually stayed the same? But it did hide sins that would only be discovered on long running sessions. A slow but somewhat reliable experience occurred. It was simpler times.

One Screen Size Fits All

Then Brandon breaks in. He remembers a time when the entire platform had but one phone, with one screen size density. You could whip together absolute positioning and it all kinda worked out. You needed to handle the “status bar is now on top as you are in a call” type exceptions, but the core components kinda took care of all that anyway. It was simpler times.

A Messy, Better World

Fast forward to today and we have a slew of devices withe various inputs and form factors. There is competition in the marketplace and user growth continues to explode.

The constraints are very different, and we have been able to learn from each other. The Web has gone mobile native, and the native app mobile platforms became more adaptive and diverse.

When you are in the thick of things it is easy to remember the past, and to remember it through rose lens glasses. You forget your complaints of that time. The early hacky SDKs and tools. The capabilities you didn’t have.

It reminds me of when people talk about the magic of the 1950s. As much as I dislike aspects of current society, we have progressed since them in many many ways. We just have to keep pushing.

Progress.

ps. A few years later, Calvin came to the same bench, sat down with Kevin and Brandon, and started to tell a tale of a simpler time for VR when the devices had constrained limitations… rinse and repeat!

Progressive Rendering: A killer (and under appreciated?) feature of the Web

March 24, 2016 Leave a Comment

I am flying home from a fantastic trip to London and India, where I had the pleasure of meeting with some fantastic engineers who are pushing hard to deliver great experiences to their users and their business.

There is nothing like getting a fresh perspective than when you spend time with different cultures, and this trip definitely got me thinking about the challenges that we face in different parts of the world, as well as the opportunities that are very much shared.

Being on 2 and 3G connections, and being on the other end of massive latencies (even when going through some LiFi) reenforces the importance of performance.

What really hit home was an inherent feature of the Web that I don’t feel like we appreciate quite fully: the progressive nature in which browsers can render content as it comes in (unless we mess it up).

Jake Archibald does a fantastic job at explaining this, and his side by side examples remind you of the difference.

With Ajax on the one hand we laughed at how painful it was to think about a world in which we destroy and create the world of our application on each page request. What was also amazing though was how good browsers got at not making it seem that bad. As you navigate around a website with the same general shell, you notice that the browser often does the right thing and it feels like the header is kinda hanging around.

The proponents of solutions such as turbolinks take that to the next level by replacing the content area (and thus keeping the shell around, with the JS and CSS all loaded and prepared). Sometimes these solutions actually feel faster than a traditional SPA. Why is that?

Often the reason is that turbolinks solution plays right into the capability of the browser to progressively render the content area. On the other hand, the SPA may be connecting to a JSON API and only updates the UI when the result set is fully loaded and passed on to render a view. Whoops, you may not have even realized it, but you bypassed the ability to render 🙁

The perception of a progressive render

When a user starts an action and slowly sees content drawing onto the screen they see momentum. Here it comes. Wait around. Progress.

Compare this to the times that you have seen a loading indicator spinning around. You know it lies. You don’t know if there is anything gumming up the works. This lack of progress can give you the nudge to switch over to something else, and there is a real chance that you won’t switch back.

A progressive render will keep the fish on the hook and is vital for engagement. It also gives the user value. The best case scenario for a progress indicator is when you trust that it is showing you reality, and it is steadily moving. For example, if you are downloading a movie and the circle is filling up steadily. This allows you to predict the end time, and as long as that time isn’t too far in the future you may hang around for it. For content on the web, rather than seeing an abstraction of what has loaded, you get to just actually see and consume it! If I go to Wikipedia, I can start reading as soon as possible, and as long as the content is streaming in faster than I can read, it is indistinguishable from having all the content ready to go.

Waiting for all of the content before it is displayed is like going back to the dark ages when you had to download the full video before you could watch it, compared to buffering and streaming so you can watch it as soon as a chunk has come in. It is night and day.

First bytes to action

Flipkart, who have created a fantastic progressive web app, have a great metric that they track: bytes to conversion. I am sure that you can come up with equivalent metrics for your business, and you should. We used to have a wave of services that would do all they could to push you to install their application. They made the leap that by doing this they would make you a loyal customer who would be worth more. The problem with this is that the causation may be wrong. Users who are loyal will have the desire to put you on their home screen and reengage. But, tricking or forcing someone to get an app there won’t necessarily make them loyal. In fact, we have shown how hard it is to get repeat visits from applications. If I had a dollar for the number of apps that I downloaded and basically never used…..

Instead of trying to force a user down the loyal line, before they may even have had a chance to check you out, you can instead take a different path. On this path you focus on delivering a great experience, and you build that loyal user. From the second that a link is shared, and I tap on it, what do you do for me. If you make be go to an app store first, you may have already lost. If I go through with the install, I am now sitting there waiting for a lot of data to come down before I can even start the experience.

If you are a publisher, monetizing content, why would you do that vs. work to deliver that content, in a way that I can start consuming it as soon as possible. Get me into the funnel asap. Don’t give me a chance to move on.

Crossing the streams

If you believe in the power of progressive rendering you need to make sure you are doing everything you can to make sure your experience is not messing up and stopping the natural flow from working. You need to be flushing data to the browser correctly. You should consider server side rendering to get HTML down as fast as possible (v.s. inlining JSON, or even worse having to make another connection to get the JSON back).

Jake has been banging the drum on how fantastic it is that we are getting the natural flow into the world of JavaScript, thanks to streams, a new API that works on top of fetch(). To handle backwards compatibility, on the browsers that don’t support it, you can also work with XHR and make sure to be able to take content out of the response as it comes in (e.g. delimiter the response in chunks of JSON that you can parse).

With streams themselves you can do all of this much more efficiently, bypassing the building of a large response in memory. This also means that you can really simplify how smart you need to be when it comes to the size of your content.

When you are dealing with slow data connections you may think about doing something like this:

renderItems(onBad2GConnection ? 5 : 50);

Seems reasonable. Send back less data on that slow connection. Then you start thinking about how you are doing to implement the onBad2GConnection piece. Do you look at connection.type? Well, that doesn’t tell you the reality of the network conditions. There is a really good chance that you get this wrong, so wouldn’t it be nicer to not have to consider that?

With streaming, there is no real harm, as the data just keeps coming back and you keep rendering it.

Something that Apps Don’t Have

Each platform has trade offs. Apps have advantages, but we don’t always talk about the good things about the Web. We often talk about the awesomeness of URLs, and how these can be shared anywhere. These links connect together the Web and allow experiences to flow, and we allow users to traverse that flow by going back and forth.

I believe that progressive rendering deserves to be up there side by side with these pockets of goodness, and we owe it to our users to come up with solutions that keep the stream moving. Apps need to download, and then get data that is used to populate a view. The Web can stream both the data and the views. On the fly.

With streams we can take this to our richer applications and allow us to efficiently deliver a great experience. Streams are yet another progressive enhancement though, and we can get streaming to work just fine on old browsers, as long as we are vigilant and don’t put up a damn.

Progress.

The Web has Evolved for Mobile

March 16, 2016 Leave a Comment

The Web is an evolving platform, and it has been consuming the abilities of mobile, which is letting it get to the next stage.

When you look at various species, you see generalists and specialists. Human’s are the ultimate generalists, the swiss army knives of animals who don’t have the sharpest teeth, yet can adapt incredibly well. Our brains give us the capacity to learn new skills and take us in very diverse directions. On the other end of the spectrum we have something like a koala beer, a creature that has specialized to the point that if a particular food isn’t available it is kaputt. There are trade offs on the spectrum of generalist and specialist, and one isn’t necessarily better than the other. If you specialize to have a strong niche that others aren’t fighting you for you can do very well, but you are beholden to that niche.

Flexible platforms are able to see the changing landscape around them and adapt to it, picking up the best ideas from other platforms and finding a way for the capabilities to fit in.

The Web has evolved many times before, and it has had various ways to test ideas. Let’s take a look at some of them.

The Document Web

The Web was born as a document platform. I was at the University of Minnesota, in the computer science department that brought us gopher (Minnesota Gopher’s and all). Oh how proud we were to deploy documentation through our gopher servers. The library met the Internet! And then came along the Web and blew it all out of the water.

The ability to publish documents with a simple format that could arbitrarily link to each other from anywhere within was of course simple and amazing. This ability unleashed the crowdsourcing of the worlds information at unprecedented levels, and the humble roots grew the abilities of these documents. New elements were added to allow for richer semantics. CSS brought much richer styling. And then Pandora’s box was opened when we added the scriptability to the client via JavaScript. Until then we were only creating dynamic content by sending it down from the server via CGI and friends. You obviously wouldn’t want to go on a server round trip whenever you moused over an element.

Other extension points were created in user agents. Thanks to plugins and ActiveX we got new parasitical evolutions of sorts such as Shockwave and Flash. They are often cast as The Bad Guy, but we learned a lot from them and baked in many of the good ideas into the Web itself. And, it was ActiveX that helped us push over the tipping point to get to the next stage….

Ajax: The Desktop Application Platform

XMLHttpRequest first came to us via ActiveX. Finally we had a way to do round trips that wasn’t as limited as frames or CGI pushing data. XHR was the hero, but it was only one of the cast of characters. We often pushed the boundaries by wrapping Flash (remember sIFR?) and the early period was a fight to deliver applications over the Web.

Why were we fighting so hard to contort this document platform when we had native applications that had full access to the power of your desktop computer? The frictionless distribution of the URL.

There is quite a difference between sending your parents to the store to get a CD with an application on it, vs. emailing them a link. Even once we got to the point where you could download a zip file, it took a long time to get the application and you were trusting your security by installing it.

We were also finding out way on what a web application should feel like. Outlook Web Access very much looked like an uncanny valley of “this is like a desktop application but it sits within the world of a browser so it isn’t quite right”. Gmail on the other hand took a very different approach, one that felt like the Web.

Quickly, Ajax became less about XHR, and more a phrase that meant “making sure you are delivering a quality user experience”. Then it faded away and once again became part of “the Web”.

The Mobile Web

Mobile finally really happened, en force. I had been hearing about it for years. At every JavaOne Sun would get up on stage and pronounce THIS as the year for mobile and J2ME. And then another year would pass. It was only with the iPhone that we saw things really kick into gear. One of the reasons was that the iPhone delivered the “full Web” to your pocket wherever you were. This is quite a feat if you think about it. Browsers were tuned for running applications and a desktop experience, and this underpowered device could run the content? Sure, pinching and zooming wasn’t an ideal experience, but you could access it!

Console Wars is a fantastic book that discusses the battles of Sega and Nintendo. One of the mistakes that Nintendo made that allowed Sega to get in the game was the lack of backwards capability in a new Nintendo device. It is often important to remember to bridge one generation to the next.

You want to be able to play your back catalog, and the iPhone played the Web.

But this is just a bridge. You also want to use all of the capabilities and power of the new platform, which allows developers to create this new content. The Web didn’t offer the right level of capabilities fast enough, and thus we got into the era of the native app ecosystem. This is where we got a lot of experimentation, and each generation added new sensors and best practices.

Innovation is still happening of course, but at this point we have learned many of the core tenants that deliver a great mobile experience. This includes system level integration so the web applications feel part of the native platform. For example, you need to be able to have access to a progressive web app right from your home screen, and it needs to be a better experience than a bookmark that opens a browser to a particular tab.

Reengagement is also key, which is why push notifications support is critical. We are seeing great results when web applications implement this feature in a way that users have control.

For example, Beyond the Rack has shared:

26% increase in average spend per visit by members arriving via a push notification
+50% repeat visits within 3 months
72% increase in time spent for users visiting via a push notification

It is also worth noting that getting web push notifications also helps on the desktop!

Service Workers is to Progressive Web Apps as XMLHttpRequest

All of this work depends on Service Workers. The Web was built assuming that you were jacked in, and it was vital to have a capability that would allow you to take control of events outside of a running Web page. The control that you get, mapped to new capabilities such as streams, enables you to deliver fantastic experiences.

I have often have orchestration services that would work to optimize and cache data based on the needs of the application. For example, when you think of a product details view, the caching SLA around the price is very different to reviews. It is fine to not show a review that JUST came in, but you can’t show an old price.

With Service Workers, you get to do this work on the client, and you can design your applications in very custom ways to eek out the best of the best. With a nice model driven design, you can send over cached versions and then postMessage updates as they come in (if appropriate).

There is still more to do, patterns to flush out, and “Google Mail|Suggest|Maps of PWA”s to ship.

The Web continues to evolve. With the rise of the app ecosystem many companies needed to ramp up and deliver there, but for many companies it is time to rebalance and deliver progressive web apps. The mobile universe is expanding in all directions, and we need to think about improving all aspects of our services.

There was a time in which we talked about Color TV before color was the norm and was subsumed into TV. The same will happen with mobile Web becoming Web.

As always, I am excited to see all of the vectors of exploration, and to see the Web continue to adapt and evolve.