Archives for January 2015

Standardization. We are seeing another go-around thanks to the enthusiasm around Docker: An open platform for distributed applications for developers and sysadmins.

Having various parties come together to look at an open view of services at this layer of abstraction is great news for developers and operators. The black box effect of having a container of stuff with a declarative way of defining how to connect it into the ship is only a Good Thing.

I wonder if we are getting a little giddy and proud of ourselves prematurely though.

We Have Been Here Before

I believe that we have seen this pattern in the past. I remember an experience that parallels this in some ways, and that is when we got Servlets and EJBs in the Java ecosystem. Before that time there were various Java servers with competing APIs and features. You would have to choose between various application servers and write to their specifications. You were pretty much locked in. There were some work arounds, and a few people would separate their logic into the world of (what would later be called) POJOs and have adapters that plugged them in. Most didn’t.

Then Jason Hunter and friends put together Servlets and we had a standard way to declare how a component as described in a piece of code would talk to its container, and the container could then take the abstract and make it physical. For example, your code would talk to “TheProductDatabase” and in XML land you would map that to the actual Oracle Database you were using behind the scenes. The standard and the container helped make that happen (along with other abstractions such as JDBC in this case).

All seemed hunky dory. Many would build large scale enterprise systems in this manner. However, did you notice that there was a promise of shared components that never really materialized? Did you notice that most people didn’t migrate between multiple application servers that often?

Having the container standardized is fantastic, but it turns out that the ship that is transporting it still really matters. Companies such as WebLogic grew and then helped BEA grow as their ship was superior. WebSphere was known as “WebFear” yet over time enough was built on top of it (e.g. portal servers) that it wasn’t about deploying to WebSphere and was more about buying the solutions.

TheServerSide.com

I was able to run a unique experiment back then. The business development team had a great idea to bring in a few bucs by showing how EJB could work across different application servers at the same time!

This lead to “TSS Announces Portability Re-Launch on WebLogic and Oracle9iAS”. What wasn’t talked about so much was the pain in making it work, and how to do so we made choices you wouldn’t really make in the real world.

To run on more than one system we went to the bare minimum. We hid a lot through sharing info at the Tangosol Coherence layer which either side could grab. This meant that we used “Bean Managed Persistence” at the entity layer. You probably didn’t have to work in the world of EJB (it proved to be a flop and overly complex for no reason other than politics) and you are happier for it. The main lesson from it all was that even with a fairly simple plugin model, getting something non-trivial working across the worlds had more side effects that you would guess.

I see that again with the world of Docker. We were in the world where Amazon was the gorilla, with very special purpose APIs for the cloud. If you couple to their systems then you are just that… coupled. As per usual, if you aren’t in the #1 spot as a vendor the answer is to standardize, and the most vocal effort is OpenStack (even though it is constantly maligned). The other approach is “if you can’t beat them, join them”, which is the path that Eucalyptus took.

A modern full stack has a ton of pieces to it these days. There are multiple front ends (mobile, Web, email, SMS, watches!) talking to backends with special systems for machine learning, search, messaging, analytics, and data stores. You have the benefit of “the cloud” which in some ways is amazing, but back at TheServerSide I was able to run the entire thing no problem on a few servers for failover. I look back very fondly to the simplicity of those times.

As Docker kicks in it enables more ships to be built with more competition. Choosing the best ship is key, as in the world of production you need to answer a lot of questions such as:

• How do I keep on top of my various services?
• What is the health of my service? (where service isn’t just a container)
• How is my overall system doing?
• Where are the bad apples and can I weed them out?
• What is my solution for fault tolerance and scalable performance?
• How do I auto-scale for performance and also cost?

I am happy to see those standard containers, but I really can’t wait to see the new ships that will be built, all with gadgets to help engineers deliver fantastic experiences to their customers.

I hope that some of the best technology wins this time. It rarely seems too, but the optimist in me wants to see it happen this time!

There has been a lot of talk on two orthogonal but related topics:

• Should we (where “we” == US but can be any country) reform our immigration program to make it easier for great engineers to get in?
• Should we embrace a distributed work force?

Paul Graham lit things up again when he posted about how 95% of the great engineers are outside of the country, so of course we should have a better way to get them in!

Some people jumped in to pivot to the other question, and focused on how we should all lean into a distributed work force as the real solution to the “we need more engineers” problem.

Of course, the answer can be yes to both. There is much to upgrade about the immigration system in this country. Beyond the macro, as an ex-pat Brit, I have felt first hand how “modern” the system is. As a manager I have also felt the pain of getting great engineers into the country and taken care of. I think we all know that there is room for improvement ☺

As a fun thought exercise I drew up the flow diagram that has you thinking about the decision points for a company (or team) when it comes to going distributed and getting talent to be local.

I quickly saw that: there are a lot of factors involved, it is more complex than you think, behind every box there are many questions to ask about yourself and your own group.

For example:

“Do they want to move?”

If you believe that there are great engineers outside of your region, and that it is hard enough to get enough of them from within the region, then you quickly get to finding real people with real needs. If you find Wendy and want to hire her, and you have a preference for local working, then does she want to move?

Many American’s feel that it is quite obvious that anyone want want to move here! Especially “Silicon Valley”. I am a fan of the area myself and chose to make the move (and I am very happy I did so), but I am not arrogant enough to realize that others may have other preferences. Some may much prefer their culture. Others have family and friends that they don’t want to leave behind. Whatever the reason, although [your region] should try its best to have an amazing environment that attracts, even if we open up the immigration rules to allow for more folks in, some of the best engineers will very much still want to be “outside”.

“Is local best?”, “Would they work well remotely?”

These questions have many sub-questions. For one, the questions are not just about the person you are looking to bring in, but are also about your environment.

How are you setup for distributed working?

Before hiring someone in a distributed environment you need to be self-aware of if they will be setup for success. In my experience I have fantastic people who work in a distributed fashion. At this point we have multiple offices as well as folks who work from home. Everyone lives on Slack, and there is enough of a mix between locations that we normally don’t have a “I feel left out” type of feeling. That tends to happen when the bulk of the team is in one location and the exceptions are remote. You have to look at the shape of each team to make sure if there will be a fit here. Beware of a poor shape.

Most of the time teams are already working with tools that work well from anywhere (Slack, Google Docs, Github, Trello, JIRA, etc). You give up the same room tax, but you gain a lot too. For the role of makers, the flexibility and lack of distractions can be huge for productivity. I worked from home for over six years and I remember that feeling of flow at home.

Can *I* work remotely?

Of course, the people need to be able to flourish in that environment. Many can love working from home and the flexibility and focus. Others need personal stimulation in a way that you can’t fake through online tools. These people can get lonely and depressed, fast. If I am interviewing someone for a role and they have never worked from home…. I worry about the risk for both of us on being the guinea pigs. It is the type of thing that *sounds* good, but you have to live it.

I remember the time where I suddenly realized that I hadn’t left my house for over 3 days straight. I then made a point of scheduling at least one thing outside of my house a day ☺

Realize that in actuality, we are all working remotely

Let’s say your day job is at the HQ. Do you ever work from home, or a coffee shop, or when on vacation? Chances are you do.

Even though you are at the main office, do you work with other folks who aren’t right there? Maybe there are but in another building on a campus so you VC anyway?

You are a remote worker with respect to the other employees.

If you want to jolt that into the company a little more, how about trying an experiment. Institute a “Slack Week” (not about slacking off 😉 where everyone has to work outside of an office (home, coffee shop, whatever is fine) and see how it feels. You may be surprised with the results.

Are there multiple “local” offices?

We have seen many patterns for distributed work. The corporate standard for awhile was the “throw it over to India/China/Eastern/[somewhere cheaper]” which rarely did well depending on the context.

Then there is the subtle change to that approach with the “let those teams do maintenance” model.

Also, if you have created multiple offices, how will you structure the teams? Will an office have a shared mission? Is the office just a “hey there are enough talented people to attract in this area, so lets just try to suck them in!” and they work on projects with other teams? There are many pros and cons to these structures, and I often see them fall to the forces of entropy.

I was at Google when the SF office was announced. Anyone who lived in the city rejoiced and wanted to be able to work there. The messaging through was very much “teams stay together”. Over time, when you have some great people with certain constraints that often relaxes: “Bob is one of our best guys. He needs to move back to his home town to take care of a family member and I don’t want to lose him!”

It feels pretty obvious that we need to work hard to enable distributed working to become increasingly productive and fun. Already there isn’t an absolute “X is better” answer, and I hope we will add more to the nuance as we improve tools and processes.

My reality is that I have a fantastic distributed team, with some local focii, and I want to be attentive to that reality!

Given my history with webOS devices I think I enjoy a good retrospective of what went wrong on the way to getting a device out to market (given the fantastic engineers and people involved) more than the average bear.

The “Under Fire” piece on the story of the Fire Phone is an interesting one.

Who knows which details are true or not. There is also more information than can ever be sleuthed by a reporter, but if we take the high level claims as truth then there is one piece that I find especially interesting, and that is around the branding:

Bezos had profound reasons for preferring a top-of-the-line smartphone. Multiple sources indicate that the premium phone represented a “repositioning of the brand away from being so utilitarian and toward becoming more of a lifestyle brand like Apple,” as the high-ranking Lab126 designer phrases it. Bezos expressed some of these sentiments himself in a memo he wrote years ago, entitled “Amazon.love.” In the memo, first revealed by journalist Brad Stone in The Everything Store, Bezos describes his vision to transform Amazon into a brand such as Apple, Nike, or Disney, which are “widely loved by their customers, and are even perceived as cool.”

Amazon is famous for not making a profit and playing a very long game. Apple is famous for building revolutionary products that consumers adore, often as art. The profit margins are huge and Apple has thus become able to have quite a war chest.

There is a burden that Apple carries though, and that is the burden of expectation. When Apple comes out with a new product it has to be revolutionary else it is a let down. From iMac to iPod to iPhone, the next thing better be amazing. What a bar for the Apple Watch when it comes out, or whatever is in the wings. The iPhone 6 is arguably the best phone ever made, but it was still “incremental” in many eyes (that didn’t stop everyone from picking one up!).

A great team would rather have a high bar to push themselves to build the best possible product. In many ways it is a true “good problem to have”, but what about if Amazon wasn’t fixated on changing the brand to be something it isn’t (yet) and instead leaned in to that brand in the short term and baby stepped its way to the future?

If Amazon is about great customer experiences with great prices then why not build that. Make a rock solid device at a fantastic price. Within the Android ecosystem there is plenty of room for this device to exist, and if the team is focused on that vs. getting cameras working to watch your head the better everyone would be for it.

Enjoy that the bar is lower than it is for Apple right now, and get a few more jumps in. The company has shown patience for the long game, so no need to put the bar too high on the first go around.

The part that would scare me the most (if true) is:

“We poured surreal amounts of money into it, yet we all thought it had no value for the customer, which was the biggest irony. Whenever anyone asked why we were doing this, the answer was, ‘Because Jeff wants it.’ No one thought the feature justified the cost to the project. No one. Absolutely no one.”

Here is where the customer-centric plot seemed to be lost, and where you see a lesson of going against your own gut due to the long shadow of a legend?

NOTE: It probably doesn’t need to be said that I have a huge amount of respect for Jeff Bezos and the people at his company for what they accomplish. This post is more for me looking for lessons.