Dion Almaer

Help Mario Reach The Next Platform!

March 28, 2023

Don’t leave him behind, or with a jump that’s just too far!

Nothing is static. The world is moving, and it’s the job of a platform to help an ecosystem evolve at the right pace.

When the pace is good, as the screen moves right, Mario sees where he needs to jump next and can time it well. It’s fun to be in the flow jumping from improvement to improvement!

When the pace is too fast, Mario feels stuck and either disappears off screen, or does a Hail Mary jump without a real chance to land on the next platform, falling into fire in bowsers castle.

Platforms need to treat the time that developers have to spend on evolving alongside us as precious. They should strive to minimize their toil, keeping a high level of trust with the developer community.

What are the keys to success here? How should we, as platform owners, drive things? This post will detail:

Understanding the use cases
Building enough of the new platform
Having everything we can to help you get there
Sharing pieces early
Starting the deprecation clock appropriately
Staying close to the platform
Do you need a new platform?

Understanding the use cases

We need to understand Mario’s needs and why it will be better for him to be on the new platform

When bringing up a v.next of a service, the platform needs an understanding of what the current version is being used for. A new version is shipping for a reason, and there should be clarity on basic questions such as:

How will developers be able to deliver the functionality they are offering?
Are there any capabilities that are not offered in the new version yet and what is the impact?
- When options are restricted, it’s obviously a different set of timing should be considered
When will replacement capabilities show up (in the cases when they do)?
What are the new capabilities that we will be bringing to developers and what will they unlock?

Seems blindingly obvious, but having deep knowledge here is far from universal, and past decisions are often lost, not allowing us to apply Chestertons Fence.

Building enough of the new platform

We want the new platform wide enough that Mario can stick the landing

With a strong understanding, the new platform starts the journey of getting built and iterating. It’s vital to make sure that we have an appropriate amount of it built out before sharing it with the developer community. With a minimal surface area, you are at risk of not finding enough information and thus ending up making large changes in the future, and developers are left touching a small part of the elephant and extrapolating the rest.

The more we can get the new version in close range, the better the chance we have of enticing Mario, and having him get across to the other side with a cheer.

When building the new platform, we should also make sure to do a good job with our layering. As we do this, Mario will not have to learn new things for each part of his journey, and will instead accrue understanding. Great layering also means that we will be able to compose our solutions better, resulting in less churn as we make changes. This should result in fewer massive migrations.

Having everything in place to help evolution

We want to give Mario jet packs and tools to make the leap

When making these platform changes, we are often placing toil on developers. Hopefully, there is much value too, but there will often be times where the changes we impose have a strong overall ecosystem value, but maybe not always the same value for the individual developer. The tragedy of the commons are real, and we can recognize this by going above and beyond with our help for developers.

What does the jet pack look like?

World class documentation on the why and the how. This is foundational, and includes great reference docs, tutorials & workshops, and samples & solutions.

World class tooling, where developers live all day long. Linters and codemods that give clear guidance and nudging on what changes are needed. With everything that it changing with development right now (e.g. AI copilots) imagine how far we could take this? Why can’t we have a future that has platform help in our developers code editors giving suggestions, and sending PRs to GitHub with changes that keep their projects up to date. If we did this right we could change the feeling from “ugh I feel like I am constantly being nagged to make some change! $PLATFORM understands that I have features to write and a business to run!!” to “Wow, $PLATFORM is helping me keep up to date and improving my app! I can see the improvements, and merchants are loving it!”

Sharing early

Show Mario a glimpse into what’s coming up in the level so he can prepare

The window can be pretty small for Mario, and it can be helpful to offer a view of what’s coming, as long as we aren’t flooding him with information (see: building enough of the platform!).

Depending on what kind of changes we are doing, we may be able to allow developers to play with the future pieces early.

Remix does a great job of this. It allows you to opt into future flags, and then when the future becomes the present, you are ready for it!

How does that work? Let’s look at an example. Remix 2 is coming out soon, but the changes and new features are coming online in a way that you can opt in your Remix 1.* application today. The way that you name routes and their mapping with the file system is changing from this to a new system that includes flat routes.

Instead of waiting for Remix 2, today I can update my app with a couple simple steps:

Tell my v1 app that I am ready to use the new feature via a simple declaration in my remix.config.js:

future: { v2_routeConvention: true }
Update my directories and files to map to the new system

When the feature is ready, communicating it could ramp up over time. You can start small and see a few early adopters find it and offer feedback. Recently the dev server started to console.log the fact that it’s ready, reaching more developers, and more feedback can flow in.

When Remix 2 ships, those who opted in will be able to delete the future flags and everything will just work. Now picture this for a larger number of flags. I can opt in and then I will be fully ready, finding myself on the next platform without even jumping… I just kept walking and got there.

Starting the deprecation clock appropriately

Don’t disintegrate the current platform too early!

You know those blocks that fall away when Mario has been stood on them for a bit? I hate those. They mean I have to think really quickly, anxiety rises, and I make mistakes and fall to my doom.

While it can be great to share information on the new platform early, as discussed, we should be careful when choosing when the clock starts ticking on deprecating the existing platform.

NOTE: A recent example of this was OpenAI deprecating the Codex API where the team maybe didn’t quite appreciate that although other APIs had somewhat transcended it, the work to make changes is real. To their credit, they got feedback and changed the deprecation by at least allowing longer access in the research program.

The clock shouldn’t start until the entire new platform is built, and we have the jetpacks ready. There have been times in which we build piece A of the new and deprecate the equivalent piece on the old. The problem is that the developer can’t actually migrate everything over, and they can become stuck.

In general we should cluster our changes and have clear times for most of our developers to do upgrades (the ones who aren’t jumping early to changes).

And we should be very careful not to end up in the situation that the meme shows above, where the new platform isn’t ready and the old one is deprecated. It’s so common to as often be the norm, and we need to fight entropy to change this.

I always somewhat appreciated the fact that I could schedule time after a major iOS SDK release to update our apps. The business understood this, and we had the space to make this happen in one chunk, and get our new app into the app store ready for the consumer releases. Contrast this with a drip, drip, drip of being asked to make small changes constantly.

Staying close to the platform

Favor the lightest abstractions that aren’t proprietary. Developers should be spending their time learning the platform, not new technology for the sake of it. For example, before creating a proprietary layout system that every developer has to learn, can we use CSS with our own special variables and styles and a sandboxed container to limit it? Or instead of creating a custom abstraction to fetch content, how about using the standard fetch(), even if you have to monkey patch it to add in specific auth, just make sure it’s well tested! No uncanny valley here please. Let developers bring their skills, and StackOverflow and ChatGPT along with them.

Do you need a new platform?

Mario would be happy to walk along and maybe take some stairs?

Before falling for second system syndrome, double and triple check that the right path forward is a new platform at all, or if there are smaller steps that can be made that over time will get Mario where he needs to be.

Respecting developers time

Let’s treat the time that developers have as a precious commodity. It actually takes work to stand still, as there is no such thing as stable. Browsers are changing. Libraries and SDKs and tools are changing, and we add to that. The more we can do to minimize it by putting in work on our side, the more leverage we get across the ecosystem. We want them spending as much time as possible on amazing features for our merchants, and jumping their way to success along the way.

🍄 Let’s a go! 🍄

/fin

I have scissors all over my house

February 20, 2023

Midjourney hallucinates scissors and houses

I used to be the type of chap who has one place for everything. The scissors are in That Drawer in the kitchen. It kinda worked, until I had to live with other humans that I didn’t have control of.

After years of fighting against the system of “one place for every thing” I went full in on the other direction, I think inspired by reading Algorithms to Live By, and Brad Fitzpatrick’s work with Perkeep. Since then, I have taken the opposite approach to a single source of object, and instead I put items in many places, ideally where they would be used. This is why I now have scissors all over my house, or screw drivers, etc. “All over” is going a lil far, I put them in spots where I think they will be used. This is akin to a CDN… caching a copy closer to where someone needs it.

Why am I talking about scissors?

I went through the exact same kind of transition in the virtual world. Where do I keep my data? I would try to centralize it as much as possible. E.g. Google Drive as a source of truth, and then split off for types of content that weren’t a fit (e.g. I used Asana as a central database for a long time, and 1Password for passwords, and Active Recall for learning, and Type.ai for writing, etc).l

After years of trying new systems, and migrating data, I took the other approach:

I embrace the fact that one system won’t be perfect, for now, and especially the future
I don’t worry about migrating data as I try new things. I jumped around with Roam, Obsidian, Logseq as an example. Once a lil more settled, I may then do some migration
I favor products where I can get to the data (yay owning your data)
I favor products where there are strong integrations. Instead of a central merge, I can then connect all of the things and have the data show up in all of the places

But how about finding things? Integrating into one search to rule them all is vital when you have data in various spots. I’m hopeful for products such as Needl (I need more integrations and a SDK to integrate with). My latest foray here is, importing all of my second brain into my own local Polymath with access control.

Now I can use natural language to get semantic search results, each with links that poke me to where the data is. Often it’s in multiple spots so I can choose what I want to open up to see and use that data.

There is so much opportunity for us to get to a collective, with integrations, and allows us to evolve and connect our data.

GenAI: Lessons working with LLMs

February 14, 2023

Creativity & Constraints, Foundations & Flywheels

The developer community is buzzing around the new world of LLMs. Roadmaps for the year are getting ripped up one month in, and there is a whole lot of tinkering… and I love the smell of tinkering.

At Shopify we shared a new Winter Edition, which packaged up 100+ features for merchants and developers. Some of the launches had a lil Shopify Magic in them, using LLMs to make life better for our users.

I had a lot of fun, shipping something for developers that used LLMs, and I thought I would write about a few things that I learned going through the process of getting to shipping.

UI for mock.shop — The mock.shop homepage

What did we ship? mock.shop

We want to make it as easy as possible for developers to learn and explore commerce, by playing. We wanted to take as much friction as possible from being able to explore a commerce data model, and build a custom frontend to show off your frontend.

This is where mock.shop comes in, it sits in front of a Shopify store, but doesn’t require you to create one yourself. Just start playing with it and hitting it directly!

One thing we have heard from some developers is that they are new to GraphQL and/or new to the particulars of the commerce domain. We show examples, and the GraphQL and code examples of how to work with it, but could we go even further?

Generate query with AI

What if you could just use your words and ask us to generate the GraphQL for you? That’s exactly what we did. And here’s what we learned…

Foundations & Flywheels

We used OpenAI for this work, and when working with LLMs you are working with a black box. While GPT3 had some knowledge of GraphQL, and Shopify, it’s knowledge was out dated and often wrong. Out of the box you are working with anything that the model has sucked up, and you can’t trust this data at all.

You need to do all you can to feed the black box information so that it can come up with the best results. Given the black box, you will need to experiment and keep poking it to see if you are making it better or worse.

Here are some of the foundational things that we did:

Feed it the best input

Gather all of information that you think will nudge the model in the right direction. In our case we gathered the GraphQL schema (SDL) for the Shopify storefront APIs, and then a bunch of good examples. With these in hand, we would chunk them up and create OpenAI embeddings from them. You end up with a library of these embeddings, which are vectors that represent the chunks of text.

With these embeddings we can take user queries (eg. “Get me 7 of the most recent products”), get an embedding from that query, and then look for similar embeddings from the library that you have created. Those will contain snippets such as the schema for the products GraphQL section, and some of the good examples that work with products. We call this context and you will pass that to the OpenAI completions endpoint as part of a prompt.

Customize the prompt

You will want to play with prompts that result in the right kind of output for your use case. In our case we are looking for the black box to not just start completing with sentences, but rather give back valid GraphQL.

You end up with a prompt such as:Answer the question as truthfully as possible using the provided context, and if don’t have the answer, say “I don’t know”.\nContext:\n${context}\n\nQuestion:\nWhat is a Shopify GraphQL query, formatted with tabs, for: ${query}\n\nAnswer:

You can see how the prompt is:

Politely asking for the answer to be truthful
Nudging for the answer to be tied to the given context (from the embeddings) vs. making it up from full cloth, and saying that it’s ok to say “I don’t know”!
Asking for a formatted GraphQL query

One other way that we try to stop any hallucinating from the model is via setting the temperature to 0 when we make the completion call:What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

It’s quite funny to see how we do everything to try to get the model to speak the truth with this type of use case!

Feedback and Flywheels

Now it’s time for the flywheels to kick in. You want to keep feeding the context with high quality examples, sometimes show what NOT to do, play with different prompts, and start getting feedback.

You will see lots of examples where users are asked for feedback. E.g. in support systems and documentation: did this help? is it accurate? To train the model as best as possible, you can look for ways to get this information from the experts (humans!) and feed it on back, as well as simply tracking what your users are asking for and how well you are acting on those needs!

Creativity & Constraints

We have the foundations in place, and the quality of data will improve through the flywheels. Now it’s time to get more constrained. We are doing all we can to nudge for truth, but you can’t trust these things, so what guardrails should you put in place?

We really want the GraphQL that we show to be valid, so… how about we do some validation?

We take the GraphQL that comes back and we can do a couple things:

We would tweak it, when possible, to place valid IDs and content, for the given dataset that we have in the mock.shop instance.
Validate the GraphQL to make sure the syntax is correct
Run it against the mock.shop, since we have real IDs, and show the results to the user!

You can’t assume anything, so you often will have to have a guard step once you get results.

ChatGPT vs. Stockfish

There was a lot of hubbub when someone pit ChatGPT vs. Stockfish in a game of chess. Many used it as a way to laugh at ChatGPT. This thing is crazy! It did all kinds of invalid moves! No doy! You have to assume that and build systems to tame it… a chess engine wouldn’t allow invalid moves.

Defensive

You have to be incredibly defensive. You are poking a brain with electrodes. It comes out with amazing things, but you can’t trust everything that comes back. Making remote calls to OpenAI itself is flaky, and often goes down.

Now only will you be checking for timeouts and errors in results, but you should consider a feature flag toggle. In the case of mock.shop, the tool is usable without any of the AI features. They are progressive enhancements to the product.

We can add checks to automatically turn it off if something really bad is happening with OpenAI. Marry both:

const openAIStatusRequest = fetch("https://status.openai.com/api/v2/status.json");

and check the results for the type of incident:

openAIStatus.status.indicator === "major"

It’s incredibly fun, getting creative with how you can use the power of LLMs, which are getting better and faster all the time. The black box nature can be frustrating at times, but it’s worth it.

I hope you are having some fun tinkering!

There are so many helpful libraries out there. I have been working with some friends on Polymath to make it simple to import and create the libraries, as well as query it all.

« Previous Page