Dion Almaer

Software, Development, Products

QA

QA Engineers Are Underrated

Leave a Comment

I wrote about how project managers are underrated and got some interesting feedback. There were probably three general buckets of response:

  • “Oh man, I didn’t get it until I had $INSERT_GREAT_PM on the team and now I wouldn’t change them for the world”
  • “I have never gotten value from a project manager”
  • “Engineers should manage the projects themselves. By having a project manager the engineering team cedes that responsibility and you end up with bad outcomes. I would rather have an intern engineer” (this always came from an engineer ;)

Whenever you have a role that comes with a responsibility there is a debate around “should this be concentrated in someone with that role, or should the team share it all so they have full ownership”. I have sympathy for this balance. I remember working with someone who was the perfect maker. He was the best engineer I knew, as well as being the best visual designer (and product manager and …). He was amazing. Having talent density like that meant that teams could be very small and the communication cost would be minimal. The side effect was fantastic productivity and the product turned out great due to his abilities.

One person can hold a lot together. Look at what Linus pulled off back in the day. Over time though it can become a limitation (look at the Linux community and what they accomplish now). You give up communication (and implicit to this, shared understanding) costs but you gain multiple perspectives and more ideas. If you can pull a team together that can work well, it can scale, take on more challenges, and you end up with better solutions.

Quality

When I was writing my thoughts on project managers, I had other supporting roles in my mind too. At the top of the list were QA (or QE) engineers.

Just as there are plenty of bad ones (like project management book keepers) there are a few great ones that have top engineers in awe.

The best teams that I have worked with have top engine who deeply care about quality. Sometimes that comes directly from the teams, but other times that comes with great QA help. These folk are deep in the weeds and they have that knack for finding the edgiest of edge cases.

If you take a world such as mobile, they know more about the various devices and platforms out there than anyone else on the team. While the engineers are solving engineering problems and writing a boat load of code, they can focus on quality. Imagine what a great person could do with that focus and time? A lot.

Ordered JSON?

When I work with a new team, if I am not getting great answers from engineers I go meet with the QA team. They tend to be the ones that know what is really going on in the systems.

I have to tell you a quick anecdote that had me realize how much time is being wasted in corporate IT land.

My team was working on the integration with a very legacy system. Our mobile application consumed a simple JSON endpoint, and it was working well against our mock servers. Once we got to finally integrate with the real systems we got an HTTP error with some internal error code 203948596. It took a week to be told that this error code was the system telling us that the JSON was sent by us in the wrong order. Hmm. What did they mean by that? JSON keys are a hash? After some digging we found that their backend (DataPower on top of WebSphere :/) was indeed setup to enforce an order. They wanted us to send key1:value1, key2:value2. We didn’t care about the order and were just serializing our an object into the JSON via the platform (Android, iOS, etc).

We asked for them to just ignore the order, but we heard back (via a good ole LOE that involves a lot of multiplication) that it would be 9 man months of development to change the behavior. Nine months. Wooah. I guess we could hand write our JSON serialization, but you just know that would be brittle. Something didn’t smell right, so I jumped on a plane to meet the team on the other end.

It turned out that the team had a “dev lead” who was managing the project where all of the engineers were off-shore. I quickly found someone from QA in a closed lab and asked them to show me what they do to test the system. Fast forward a bit and I was able to grab the controls of the DataPower GUI (an enterprise beauty of course!) and managed to find some advanced settings where there was a check box for “Validate order of JSON?”. Uncheck. Retry. All worked. One checkbox almost cost folks months or a buggy slew of new code. Then I sat back and thought “how often does this happen every day?” and I went out for a beer, which I drank as I wept over the OrderedJSON documentation.

The Real Spec

The great QA teams often hold the true spec. It isn’t the pretty one that holds the theory, it is the one that holds the truth of the various systems. The gnarly differences between QA1 and PQA3.

They have the slew of tests that go beyond the happy path and a couple of edges. To do this they are often writing a lot of code themselves of course. We aren’t talking about manual wranglers here.

They are the true holders of the use cases. The UX team has some red lines and notes. The product team put together some business scenarios. The engineering team implemented a bunch of this work, and then the QA team went in and did some really hard work to get the full coverage. They went back to the members of the team to wrestle these edge cases to the ground.

“But really, if the legacy Black Card members apply this gift card does their old 5% apply before or after? Oh, and they are in Vegas when this happens which has the obscure tax law…..”

The good ones enjoy covering the bases. They enjoy doing everything they can to find the bugs. They know the importance of catching them early because they have seen how much longer it takes to fix bugs that linger. At Palm they did research to show that it takes 24 times as long to fix a bug that was found a day later versus an hour later. Yikes, that is a real impact.

A Quality Culture

Finally, the great ones manage to build a culture of quality throughout the team. The team doesn’t rest on their laurels, knowing that Praneeth will find the bug later anyway, they instead do everything they can to make sure the bug never finds its way into their system.

So, I salute you… you are engineers that show true quality.

Others in the series:

The right thing to do, is the right thing to do.