Passing through the levels of hell to get to insights through Data

From ignorance, to manual pain, to being owned by a tool, to helpful, to automation, to insightful

The “Internet of things”. It’s going to be big, as they say. I am excited about that world, but also fear that we have some stages of hell to go through until we get to the promised land.

Stage 0: Ignorance

It is said to be bliss, and I understand why. Before I wanted to track any data my life was so much easier. I could just Be. As I have jumped into other stages, there are times that I come back to this stage, wondering if it isn’t so bad after all.

Stage 1: Manual

For so much data, the tasks are so freaking manual. On the one hand there is an input coming from some device which is great, but I quickly find that I want to do something useful that is slightly off the beaten happy path, and my life becomes full of manual pain.

Stage 2: The Data Owning You

I fell into the trap of the data changing my behavior in negative ways. My case was to do with health. I was using a Nike FuelBand, and I was trying to optimize the fuel. Just as my body had been tricking me to get sugar into my system by putting thoughts into my mind such as: “You are thirsty. Drink Soda!” I found myself going for a walk rather than a bike ride (or even worse… lifting weights) just because I wanted The Fuel.

I see this time and time again. The data had gone beyond a useful input, and my world had been flipped to be all about getting a particular output, and when the calibration is incorrect bad things happen. I am looking for this miscalibration whereever I can now, and I fight to be at peace with the data showing me one input, and not obsessing over it when I know it is incomplete.

Stage 3: Helpfulness

When you are training a new system, you don’t have to jump to automation but can rather have the youngling start to feed you information.

I am looking forward to seeing more of this. For example, back to the health tracking scenario, as I use a tool to cultivate habits such as “Workout” using Lift say, it would be nice if a device on my person (e.g. my phone, or FitBit) worked out that I was at the gym and at least put that checkmark in a “I think you did this” stage. I could then correct it. When I look at my habits, there are many more of these.

Stage 4: Automated Connectedness

For example, one habit that I work on is “Learn Spanish”. Rather than tapping on a button to say “yup, did it!” I wish that Lift would be aware of the fact that I used DuoLingo today and marked it off the list for me.

When you get more connectivity between data sources it can then do so much more. Rather than the simple “check!”, fill out the comment with info on what was actually worked on.

When it comes to checking off the “Workout” habit, I put in what I actually did. Ideally this new world would know to put in the run info from Nike+ Running, or grab the work out info that I put into a Google Doc to track progress.

This is where I start to wonder about how data will be shared. Using IFTTT to wire things together is a great service, but far from ideal (especially for the average Joe). The trust model needs to get worked out. As soon as you give some level of access and the bits are copied over, they are out of your control. A lot of smart people are working on various models, but getting the balance of control vs. ease of use is really hard. A lot of the interesting data that you want to connect is private indeed. Think through your use cases and then noodle on what you would be willing to share. Than think about what people could reverse engineer with some of that data (e.g. your location!)

Stage 5: Insightful

Once we have truly connected data with smart automation, the data starts really working for you. It isn’t enough to be capturing data though. We have seen that with the “Big Data” movement. I now see a lot of companies filling up storage and pushing Hadoop and Cassandra, but WHERE ARE THE INSIGHTS!

Once we start to see the insights coming we will get another level of “a ha!” moments. We see see glimpses of this via services such as Google Now. As we learn that “great experience” != “flat design” and see that context and data are actually king makers, we see the fact that Google has a great role to play.

Much of what makes a great experience is the balance of output to input. The iPhone was awesome as it gave us access to the output of the Internet in a beautiful and useful form (Mobile Safari vs. Blackberry etc). The one area that some held against was the lack of a tactile keyboard, and I am still hopeful for haptics to come to play some time soon. Data and context can do magic by giving us output without manual input.

Stage 6: The Machines Take Over

Other than profit, where do we end up? Well Terminator 5 of course.

I am genuinely excited to see the data revolution kick into gear. With an open platform we will see people bring insights into our lives and the end result will hopefully be in seeing the chance to get to a higher purpose in our lives, health, and happiness.

Until then though, I am getting ready to jump through some hoops as I scale some levels of hell.