The Order of the JSON

When I read Fitz’ tweet about ordered JSON my body shuddered, as my brain was flooded with a past experience that taught me a frightening lesson on technology being used by non-technical companies.

It was a moment that had me wonder:

  • How does world not break due to technology more often
  • How much time and money is wasted due to some ignorance at some part of the development cycle?

Ok, here goes. My frightening tale around the order of the JSON. I was working on a project for a large Fortune 500 (I will protect the name of the company to protect the innocent!).

The project at hand required integrating a modern front end (mobile apps and web) with a legacy system. This isn’t legacy in a “the engineering team really wants to rewrite it in the new shiny”, but in a “there is COBOL running back there somewhere isn’t there” kinda way.

Much of the system was so old that it was hard to find anyone who knew how it actually worked, and it’s maintenance had been outsourced to some of the typical IT outsourcing companies of the time.

At first all was well. We had created mock services that spoke the protocol and we were building against it. We were dealing with pretty simple REST calls with JSON, so it wasn’t like we were all SOAP-y.

We wanted to integrate with the real systems as soon as they were ready of course, and that’s when it got fun. I remember the first time we spoke to the system an got a terse error message:

ERROR 1000294

That was it. No more info. We asked the service folks for more context, and *two weeks later* we were told “Oh, the JSON payload that you sent us was in the wrong order“.

Huh? Why would you care what order we were sending the name value pairs for this?

We asked if they could fix this and be more flexible. They said they would get back to us.

It turned out their did a “LOE analysis” (level of effort, in case you haven’t had the privilege), and came back to tell us that they would fix it, but it would take 9 people 6 months to do complete. With a straight face. In a way that signalled that this is the cost of doing business and it happens all the time.

We couldn’t wait that long, and we thought about creating a custom emitter that would indeed order the JSON. We wouldn’t want to do it on the client, as what if something changed? We didn’t have faith that this “order” would be set in stone forever.

Ok, let’s handle it on the server then…. and have continuous testing to make sure that if anything changed we would know right away. Not the end of the world I guess.
But it still didn’t feel right. I got on a plane to visit the site where this was all going down.

I mentioned that it was hard to find people who knew how these systems worked. I have previously mentioned how QA engineers are underrated and they came to my aide again here. I hunted down the QA lead and he took me on a tour of how he tests the systems.

I got to learn that we were talking to a service running IBM DataPower Gateway which sat on top of WebSphere which sat on top of the COBOL.

He was showing me the DataPower GUI, and let me play around with it. After awhile of drilling around and going through settings I got to a hidden advanced setting that was a checkbox asking:

Validate order of JSON? [X]

I unchecked it. Ran a client to post the JSON to that instance. It worked just fine.

And, then I sat back and contemplated how this type of event was probably occurring daily, all over the world, and some consulting companies were getting millions of dollars.

Finally, I was so curious about where the root of this came from, and did a search for ordered JSON and ended up on this IBM javadoc.

Of course! I am sure you have never run into scenarios like this in you time in tech, have you?

Does mobile documentation matter for developers?

Developers code on their laptops, normally docked with their mechanical keyboards and monitors, right?

As such, when it comes to developer documentation, you don’t need to be responsive, and can focus on a desktop-only set of docs.

I have heard this sentiment many times, and at first blush it makes some sense. For all the talk of mobile eating the world, a lot of the business end of things happens on the desktop Web.

Once you have desktop docs going, you look at your stats and see that you were right! A fraction of traffic is coming from mobile! Glad you didn’t waste any time on a great responsive experience! Well, hold your horses. You may have self selected your answers here.

A great developer experience includes a great documentation experience, and that includes mobile. I have seen transitions from desktop-only to responsive, as well as responsive from day one, and both have shown how important it was to go mobile, too.

Take one of the Google developer sites for example, and a recent slice:

This slice is from a campaign, so bias, but directionally similar

Why is mobile such a big deal?

All documentation isn’t equal. There are a variety of reasons why someone ends up in your docs. The Web is a viral platform due to the sharing of URLs through all sorts of mediums (via Email, Twitter, Slack, etc). People are increasingly accessing these mediums via mobile, and are tapping in to read from there. I often read a lot of developer-oriented content on mobile, and will save it away and often revisit on desktop.

I also do a lot of memorization through mobile, using space repetition systems. A common task during the day will be saving some developer knowledge into my system.

This is somewhat akin to customers who browse for products on mobile and finish the transaction on desktop.

Beyond documentation

It has been fantastic to see the trend of developer documentation being seen as putting the manual online with some hyperlinks, and instead using the medium to add a lot of interactivity to help you learn.

The best way to learn is to do, and it is important to be able to feel how things work and play with them. We have embedded Glitch into web.dev as an example of bringing you tight playgrounds to explore, and remix for your own experimentation.

How do you want this type of content to be approachable on mobile vs. desktop?

The other important realization for me has been integration. Developers often end up in your docs via search, and they rarely probably go to your root domain and just read read read.

Take time to look in the Search Console to see your analytics to see how users are getting to you, and make sure that you have the content they need. You need a base of reference documentation, and then can walk up the chain to use cases, and finally verticals.

Look for other areas of integration.

  • Where do your developers live?
  • Their IDE? And certain plugins?
  • Do you want to reach them on StackOverflow or Glitch?
  • Do your error messages link out to documentation to help?
  • Do your docs work offline? (e.g. if someone is reading on the subway to work, or generally stuck in poor and variable connectivity)
  • Do you include code from source files that you have tests running against to make sure that a copy paste works?
  • Have you built a connection between the tools and your documentation so it can be surfaced inline?

There is so much to do, and while “docs” are a first class citizen, it is also important to recognize that good tech writers are still under-rated!

One browser to rule them all?

Sizzy is the latest custom browser for a particular niche, this time design. Blisk is another that targets developer use cases. Ironically, it’s tagline is “One browser to rule them all” when, in fact, maybe it paints an alternative picture.

Time for an Eclipse

I remember a trip that Ben and I took to Ottawa. It was in the winter, and happened to occur at the same time as a visit from Obama (so, you know, in the good old days ;). We were visiting the core team at IBM behind Eclipse (and SWT and OSGI and ….) such as Steve Northover. The topic was web based IDEs, since we were working on Bespin and they were building Orion.

They were great to talk too, because they have so much experience in the world of tooling, having worked on them for decades.

One of the lessons I learned was that Eclipse was originally a foundation. If you look at the layers: SWT as a native widget toolkit, OSGI as a way to dynamically modularize, and Eclipse as the IDE bits…. you had a way to customize and build an IDE. Maximum flexibility.

But many Java developers downloaded Eclipse and used it. Many appreciated it, but many also found it far too confusing (Perspectives?). The IDE platform was being used as the end product. Since then, custom IDEs have been build with Eclipse, using the power of the platform but delivering a custom and opinionated view for it’s users.

I reflected on this as I played with niche browsers and wondered….

What if we had more browsers in our lives, not less?

What most think of as their “browser” is the Web platform with a product surrounding, and integrating with it. The main feel of the browser hasn’t felt that different for awhile. You still see the URL bar up top, still use tabs, etc.

A browser expected to be baked into an operating system, but we still see new browsers looking to innovate and find their own niches, such as funding models and verticals. So far, the browser itself feels familiar.

Unless you consider the “app browsers”. The apps that deeply embed the Web as a core part of their experience, such as Twitter, Facebook, Google’s Search app, and on and on. These have taken a particular view that is less of a blank slate that allows you to put in a URL (well, or search, or leap from a new tab page that has a feed too… ok there is some innovation there!).

Imagine how a browser would be different if it was oriented in a particular way? E.g. a developer browser that didn’t focus on speed for users, but on getting you as much data to help you debug and develop?

Just as with Eclipse, there are many ways to get there. You can customize through plugins, or you can have a seperate distro that is fully setup and ready.

It does make me wonder though…. can we find a new bar that allows for much more innovation in the product space, while we all work together to push the web platform forward, in a way where developers can ship to reach as many users as possible.

What would you like to see?