Dion Almaer

Software, Development, Products

UX

Stop painting and have a Meaningful Interaction with me!

Leave a Comment


When it comes to performance, what do we measure? Our focus has changed over time as we get more of an understanding on the importance of perceived performance vs. some absolute “the bytes came down in X seconds and the view rendered in Y”.

We had a lot of performance related talks at Google I/O, and it was fun to see some of this evolution. There are a series of tricks at play, where you keep the user engaged by showing them progress versus a binary blank view becoming fully available and interactive:

  • As developers we know that loading indicators are lies, but some of our users still buy them
  • We can get something painted ASAP (e.g. a shell) and then spinner until other content shows up
  • We can show fake UI component holders that are then filled in later
  • We can show snapshots of past content (sometimes literally a prior screenshot) and replace when new data comes in
  • We can show the visible content in the viewport and lazily load below the fold (AMP does this and much much more… learn what they do here and how they prerender)
  • We can load tiny images scaled up, and then replace the full images when downloaded (e.g. what Medium does with its images).

There are a lot of tools such as these to give us some slight of hand, and they work especially well when we can progress through the stages. I find that they tend to break down the most when there is a huge delay in one step: e.g. the shell loaded pretty quickly but most of the page is a huge spinner going on and on and you can’t trust if anything is happening.

Recently, we spent a lot of time measuring the time to first paint, which is often great, but it can cause you to trick yourself into thinking you have done a great job when all you have done is sped up one step, and you could have actually slowed the illusion and done harm.

Paul Lewis shared a visualization of how various approaches can differ here:

This brings us to the new topic of the day: Time to First Meaningful Interaction.

Alex talks about the “time to interactive” measure at length in a fantastic talk, and it is a key subset.

Search for meaningful interaction, and you will see links to a lot of therapy and tips for how to communicate with others. Compared to first paint it is much vaguer and requires you to think through the details of each view and what interactions are meaningful. They may differ based on the type of user, or where they are coming from in a flow, or the time of year. This is the type of thinking that you go through when you are building your UX, not just your performance. It is strategic as well as tactical.

The answers to these questions may change your approach to how you deliver a given view or application.

For example, in a transactional system that doesn’t have a lot of content to consume, you will very much want to wire up application so the user can click that button asap, and you will wait to minimize the time between “I see the button that I need to click on!” and “Why doesn’t it do anything when I tap on it?”

On the other hand, if you have a content heavy site where a user will want to spend some time taking everything in before their next interaction with the application itself, then you will be rushing to paint valuable content and allow them to scroll around to read that content. The actions and other items can lazily load much later.

Often there will be a mix and match. When you get to a product page on an e-commerce application you can quickly load information such as description and price, and load other info later such as reviews and secondary details (dimensions etc).

The right choice here will be the difference between frustration and showing the right level of progress so the user can keep in some sort of flow in your experience.

This is why I am excited for us to search for meaning, rather than blindly rush to paint, and how the Web now has the capabilities to deliver reliable resilient performance.

DuoLingo: a (better than!) text book approach to practical spaced repetition

Leave a Comment

I am a big believer in spaced repetition as a scientifically proven method to help you learn both effectively and efficiently.


There are several implementations, and one of the most basic and famous ones is the Leitner system, which is a great way to explain the point. Rather than going through a series of flash cards in serial, repeating each of them again and again, why not consider whether you got the question correct or not? When you get the correct answer the question goes to the next level of boxes, which isn’t gone through as frequently. Knowledge that you grok naturally goes further back in the line and you don’t have to be quizzed on it as frequently. The minute you get it wrong however it comes back up.


The most efficient system will know when to ask you so perfectly that you will struggle to answer, but will eventually come up with the right answer. This may not be exactly what you want however, as you may want some knowledge at the tip of your fingers.

Beautiful software back in the day!

There has been a lot of work on various algorithms, and computer systems have been developed to further the cause. The prototypical example of this is the SuperMemo software. Pavel Janeka not only built and iterated on this software over decades, but he published information on each algorithm, and then showed what seemed to work well and what didn’t.

Software such as the original SuperMemo, and the currently popular open source Anki, rely on you to rate how well you knew the answer, to feed the right information back to the system.

Was the answer immediately available in your memory? Did it take a sec? Did you have to really think about it? Once you saw the answer did you think “oh crap, right!” or was it a “huh, really? I had no idea!”. These could have different effects on when to next ask you the question.

A common misunderstanding in learning something is that it is ideal to know the answer immediately. Instead, it has been shown that for long term learning you want to have to think and generate the answer. This is known as the generation effect. Fantastic studies have shown that subtle changes to how you quiz have large effects. For example, simply asking a subject to fill in a word’s missing letters resulted in better memory of the word (crosswords anyone?).

There has been a backlash against standardized testing, but the big mistake is in how we focus our quizzing on assessment rather than using them as a fantastic learning tool.

If you are interested in lifelong learning, or how to help your kids not get stuck in the trap where we think we are learning but in fact we are just binging information that we will forget a week later, check out Make It Stick.

Where DuoLingo Comes In

With all of that context we come back to the product at hand. Once you learn about space repetition you quickly see how DuoLingo offers a user experience that hits the research without the user haven’t to answer how well they knew something (which can be flawed in of itself!).

Let’s look at the different types of question and answers and how the system gathers information.

Choosing Pairs

This type of question offers words in two languages that you need to pair up. If you get one wrong you aren’t finished, but the system can:

  • Remember which words you may have trouble with (I say “may” because I have fat fingered in the past)
  • Make sure to quiz you on some of these items again sooner than if you nailed them
  • Take the time to answer into account. If you are quickly matching that tells you that it is tip of the tongue whereas a more labored approach may mean that you are doing more generation of answers (or you may have a distraction or had to go to the bathroom…)

Translation: Typing or Picking


When asked to translate something into your language of choice there are a few pieces of information available:

  • Some words have dotted underlines that you can tap on for a hint. The act of doing this shows the system that it wasn’t top of mind
  • If you are struggling with typing the answer the system can convert to multiple choice to give you a last change to generate the answer. At this point the system also knows that even if you get it “right” you don’t know it as well.

Listening


When listening to the language and then typing in what you hear there is the option to repeat the phrase, and to repeat it slower. If you need to repeat it, the system may know something, but if you repeat it slowly it really knows that you didn’t quite get it.

Grouping of questions

The grouping of questions matters. If you go into a block of questions on the “past tense” then you have content that can make it a lot easier to get the right answer. You get used to the patterns again as you go through. This isn’t all bad, because it allows you to tie together this knowledge and its patterns. However, we always need to remember why we are trying to learn something and get as close to that environment as possible. We probably won’t be stuck in conversations that are limited to these groupings.

“We practice as we want to play, so we can play like we do in practice.”

The Chicago Blackhawks famously made changes to how they practiced ice hockey. They got away from their regular blocks “on Thursday’s at 2pm we do passing like this…” and randomized a lot more. Just when they were getting into the flow they would switch. At first the players hated it and they thought they won’t progressing, but this shows you one of the frustrating tricks… we aren’t good judges on how we are progressing. When we cram we feel confident that we are learning, but this is because it is shallow learning. We are familiar with the content, but this doesn’t mean that we know it.


DuoLingo has another option to allow for this, the “Practice Weak Skills” button that will feed you questions across the board rather than just in one of the groups. It won’t be totally random, and you will still get to fire neurons close to each other.

There is a slew of research on how we learn, and it is criminal how little we put into practice. It is a real pleasure to have seen DuoLingo iterate over the years that I have been using it, and seeing how it applies this research in clear usable ways.

With systems like DuoLingo we can measure large populations learning, in this case languages. We will be able to tweak our learning and prove out hypothesizes. We will be able to better personalize these learnings to you the learner. I can’t wait to see how we progress.

Have you seen other examples of spaced repetition or other practical implementations of learning research?

A Users Hierarchy of Digital Needs

Leave a Comment


There is some great discussion in the community about the importance of User Experience compared to Developer Experience. Nolan Lawson added a fun analogy to the mix with his piece on composers and audiences, reiterating the fact that users don’t benefit directly from our naval gazing.

Of course, the story isn’t “UX vs. DX!”. It isn’t an outright war between the two, and in fact they can work well together. Unfortunately it is easy to get the balance in the force wrong.

Jake Archibald had another great post on the new streams API for the Web. One of his examples talked about flow control, and how this new capability allows the streams and the platform to work hand in glove:

Imagine we were using streams to download and display a video. If we can download and decode 200 frames of video per second, but only want to display 24 frames a second, we could end up with a huge backlog of decoded frames and run out of memory.

This is where flow control comes in. The stream that’s handling the rendering is pulling frames from the decoder stream 24 times a second. The decoder notices that it’s producing frames faster than they’re being read, and slows down. The network stream notices that it’s fetching data faster than it’s being read by the decoder, and slows the download.

I content that we may have a backlog issue where the communities focus may not be in flow with the platform.

The Hierarchy Of Needs

Back to the diagram above. Here are the pieces of the pyramid and how they fit together in my mind.

Something Worth Doing

First of all, we should be building something that users need or want to do. This doesn’t mean that it has to cure disease, or only be something of social good, it just means that people see a use. Most startups fail because the timing of this use, a lack of marketing it, or the product not being good enough (see below) causes the rest to be moot.

This is the base. Few want to waste time building something that no one cares about. We want to solve problems and should focus here first.

Is Our Platform Capable

I would love to teleport anywhere in the world. Currently we don’t have a platform that supports that however. We next need to map the problems that we see into a solution that can actually work for our users today (ignoring R&D and academia).

We also need to work with our platforms to help prioritize the capabilities of the future. As their consumers, we need to help inform them as they may not be consumers themselves. This is an example of flow control.

  • If the platform builds features that aren’t used then there is a bad backlog.
  • If the platform doesn’t build features fast enough then we are left to our own devices, which could mean idle hands or leaving the platform.

I believe that one reason that we get a lot of DX tinkering in a community is when the platform isn’t keeping up. There are other reasons for this too however:

  • We are in an experimental phase where we have new capabilities that we are pushing and stretching
  • Productivity is a bottleneck (see below)

Can Our Users Actually Do It

Now we have the capability for a solution, can we design something that works for real humans. This is where the user experience kicks in to gear. It doesn’t matter if your platform has amazing capabilities if they aren’t wired up in a way that makes them usable.

Can It Perform Well Enough For Them To Keep Doing It

Performance builds from here and has strong ties to the usability and capabilities pieces below. Performance can make or break you. It is often the bottleneck, for example: OCR, VR, speech recognition, AI, etc. We have envisioned how computers could help us in some many places where we have needed R&D across many vectors to make them usable and performant.

Can Developers Fix and Iterate

If we can in theory build a great performing user experience, we then need to make it work in practice. It needs to not take a life time to build features. We need to be able to debug and fix issues in short order. You shouldn’t need to be an enlightened guru to be able to eek out the performance out of the platform, there should be a clear well trodden path.

You can see how all of these fit together and are important. What is the bottleneck for what you are trying to build? It may vary.

We need to work the flywheel across all of these, and when we do so great things happen. Our mobile users have new needs that we need to solve. We need to build offline first experiences that perform well for them and allow them to get their tasks done. We need to have developer ergonomics that allow developers to easily build out these features and keep iterating to solve more problems.

From problems to solution ideas to platform capabilities to user experience to developer experience. We walk up and down the stack all day. One learns from the other. The more we can do to get a reactive well oiled machine, the more we can get done.

This is why I think it isn’t UX vs. DX. This is why I think we need to work together better than ever. Our users are counting on us to do so.

The right thing to do, is the right thing to do.