Archives for February 2016

Chaos Monkey Your Team

February 11, 2016 Leave a Comment

Chaos Monkey People: every week, one randomly-selected person must take the whole week off regardless of their current work. Org must adapt.

— asolove (@asolove) February 10, 2016

Chaos Monkey is a tool that Netflix created to turn the volume on system testing to 11. If it is vital for you to be fault tolerant and flexible, you need to prove that it the case, and the only real way to do this is to break things in production and see what happens.

There is obviously a cost to making something fault tolerant, and even more so to run these tests and keep tweaking the entire system, but this cost hedging the risk of not doing so.

When Adam Solove tweeted about “Chaos Monkey People” it was funny, but it hit a nerve. It may seem crazy to slow down teams in this way. We have so much that we have to do for our users! Features! Bugs! However it can also be argued that this is short term thinking.

If you are focused on the long term stability of your company, or team, then doing something like this may not be so crazy.

It also has interesting side effects. Beyond having random people just leave (although more vacation would actually be good imo!) there is also the notion of switching roles.

By switching within a discipline you can learn new skills (e.g. front-end / server side engineering) and that also holds across disciplines. What can be truly magical is the effect on empathy. Have you ever seen people who think another role is easier? Or another team is slower than they should be? By switching things up you get to see things from the other side and I would very much bet that your mind will change in some way (it may solidify a thought, but at least there is more data behind it!)

If a team has more empathy, they will work better together. Trust and respect has more of a chance of kicking in, and this will have a huge effect.

By creating these opportunities not only do you spread out knowledge, but you may also find hidden talent.

Even if you don’t introduce the mini bus factor of Chaos Monkeying your team, it is well worthwhile thinking about how your team can be setup for long term success.

Running up and down the Web stack

February 8, 2016 Leave a Comment

As my mind shifted into consciousness on waking in London this morning, I noticed a dream where mice were running and up and down keys on a piano.

I have been thinking about how nice it is to see the Web platform evolve in both directions up and down the stack.

In the beginning there was HTML to define documents. Then we added styling via CSS, and dynamic behavior via <script>.

We tended to focus a solution in one of the worlds, and there were a few links between the worlds.

Animation

At some point we wanted to move things around, and the only way to do so was to take that script thing and change the location of the contain of the thing you want to move.

That was sloooooow. How about if we can declare animations and let the browser do all of the work? And then we ended up with CSS transitions and animations. GPU to the rescue.

But, what if we want to do some kind of movement that doesn’t fall into the auspices of the CSS capability? Ugh.

You want the best of all worlds. For common ideas that you can declare, you can CSS away and be done, however if you need something more wouldn’t it be nice to script on top of the performant browser engine rather than trying to do everything yourself (and slow)?

We are getting to that place. With Web Animations we have had the unification of declarative and imperative for awhile, and thanks to work coming out of Houdini we will be able to go through even more of an escape hatch to make sure to hit maximum polish.

For example, with a compositor worker you can ask the browser to keep you in the loop on certain DOM elements. This can allow you to keep elements nicely in sync so you can do keep 60fps as your UI flows around like this:

Service Workers

How do you want to work with service workers? Depending on your use case and how you like to work, you may prefer registering a service worker through your apps javascript:

navigator.serviceWorker.register(‘/sw.js’).then(function() {
// you now control all pages on the origin!
});

Or you may want to declare the service worker from some HTML:

<link rel=”serviceworker” href=”/serviceworker.js”>

There are many other areas where this comes up. If the platform has the capability to create a custom element, it is nice to be able to register my own element both declaratively and imperatively.

As we continue to extend the Web, I like what I am seeing with respect to taking care of: core platform capability to how to access it to how to make it very productive for developers.

Streams is a fantastic, latest example of how a team will be able to go the extra mile for their users:

Make the common tasks trivial, and make custom tasks possible.

DuoLingo: a (better than!) text book approach to practical spaced repetition

February 1, 2016 Leave a Comment

I am a big believer in spaced repetition as a scientifically proven method to help you learn both effectively and efficiently.

There are several implementations, and one of the most basic and famous ones is the Leitner system, which is a great way to explain the point. Rather than going through a series of flash cards in serial, repeating each of them again and again, why not consider whether you got the question correct or not? When you get the correct answer the question goes to the next level of boxes, which isn’t gone through as frequently. Knowledge that you grok naturally goes further back in the line and you don’t have to be quizzed on it as frequently. The minute you get it wrong however it comes back up.

The most efficient system will know when to ask you so perfectly that you will struggle to answer, but will eventually come up with the right answer. This may not be exactly what you want however, as you may want some knowledge at the tip of your fingers.

There has been a lot of work on various algorithms, and computer systems have been developed to further the cause. The prototypical example of this is the SuperMemo software. Pavel Janeka not only built and iterated on this software over decades, but he published information on each algorithm, and then showed what seemed to work well and what didn’t.

Software such as the original SuperMemo, and the currently popular open source Anki, rely on you to rate how well you knew the answer, to feed the right information back to the system.

Was the answer immediately available in your memory? Did it take a sec? Did you have to really think about it? Once you saw the answer did you think “oh crap, right!” or was it a “huh, really? I had no idea!”. These could have different effects on when to next ask you the question.

A common misunderstanding in learning something is that it is ideal to know the answer immediately. Instead, it has been shown that for long term learning you want to have to think and generate the answer. This is known as the generation effect. Fantastic studies have shown that subtle changes to how you quiz have large effects. For example, simply asking a subject to fill in a word’s missing letters resulted in better memory of the word (crosswords anyone?).

There has been a backlash against standardized testing, but the big mistake is in how we focus our quizzing on assessment rather than using them as a fantastic learning tool.

If you are interested in lifelong learning, or how to help your kids not get stuck in the trap where we think we are learning but in fact we are just binging information that we will forget a week later, check out Make It Stick.

Where DuoLingo Comes In

With all of that context we come back to the product at hand. Once you learn about space repetition you quickly see how DuoLingo offers a user experience that hits the research without the user haven’t to answer how well they knew something (which can be flawed in of itself!).

Let’s look at the different types of question and answers and how the system gathers information.

Choosing Pairs

This type of question offers words in two languages that you need to pair up. If you get one wrong you aren’t finished, but the system can:

Remember which words you may have trouble with (I say “may” because I have fat fingered in the past)
Make sure to quiz you on some of these items again sooner than if you nailed them
Take the time to answer into account. If you are quickly matching that tells you that it is tip of the tongue whereas a more labored approach may mean that you are doing more generation of answers (or you may have a distraction or had to go to the bathroom…)

Translation: Typing or Picking

When asked to translate something into your language of choice there are a few pieces of information available:

Some words have dotted underlines that you can tap on for a hint. The act of doing this shows the system that it wasn’t top of mind
If you are struggling with typing the answer the system can convert to multiple choice to give you a last change to generate the answer. At this point the system also knows that even if you get it “right” you don’t know it as well.

Listening

When listening to the language and then typing in what you hear there is the option to repeat the phrase, and to repeat it slower. If you need to repeat it, the system may know something, but if you repeat it slowly it really knows that you didn’t quite get it.

Grouping of questions

The grouping of questions matters. If you go into a block of questions on the “past tense” then you have content that can make it a lot easier to get the right answer. You get used to the patterns again as you go through. This isn’t all bad, because it allows you to tie together this knowledge and its patterns. However, we always need to remember why we are trying to learn something and get as close to that environment as possible. We probably won’t be stuck in conversations that are limited to these groupings.

“We practice as we want to play, so we can play like we do in practice.”

The Chicago Blackhawks famously made changes to how they practiced ice hockey. They got away from their regular blocks “on Thursday’s at 2pm we do passing like this…” and randomized a lot more. Just when they were getting into the flow they would switch. At first the players hated it and they thought they won’t progressing, but this shows you one of the frustrating tricks… we aren’t good judges on how we are progressing. When we cram we feel confident that we are learning, but this is because it is shallow learning. We are familiar with the content, but this doesn’t mean that we know it.

DuoLingo has another option to allow for this, the “Practice Weak Skills” button that will feed you questions across the board rather than just in one of the groups. It won’t be totally random, and you will still get to fire neurons close to each other.

There is a slew of research on how we learn, and it is criminal how little we put into practice. It is a real pleasure to have seen DuoLingo iterate over the years that I have been using it, and seeing how it applies this research in clear usable ways.

With systems like DuoLingo we can measure large populations learning, in this case languages. We will be able to tweak our learning and prove out hypothesizes. We will be able to better personalize these learnings to you the learner. I can’t wait to see how we progress.

Have you seen other examples of spaced repetition or other practical implementations of learning research?

« Previous Page