Dion Almaer

Software, Development, Products

Software Development

Piece Together Your Platform with Lego Blocks, Sets, and Kits

I love my layer cakes, and recently spoke about layered design systems that allow for developers to jump in at the layer that makes sense for them, and allows for maximum emergence of value.

When building a layered platform, I often think of the world of Lego and how utterly fun and creative that universe is. I aspire to enabling developers in the same way.

I mentally split things up into the concepts of: blocks, sets, and kits.

A pile of LEGO blocks

Blocks

Lego blocks are the base level primitives that exist for the platform. They offer clear capabilities, and have interfaces that are as universal as possible.

On the interface side you have the way the the tubes on the bottom interlock with the studs on top of other bricks. This standard allows the composability of the majority of the blocks. It’s all about the interlock and the spacing.

For capabilities, you get the specialization on top of these interfaces. Think of the engines that can be connected to the car systems to make them drivable. That engine can also be composed in a multitude of ways to deliver force for many ideas.

In our world of software, we have the same thing, with interface glue such as props with React, JSON for formats, HTTP for networks, and so on. Then when you look at a platform such as Cloudflare, you see that these are composed with infrastructure blocks such as D1 for databases that speak the language of Workers, R2 for distributed object storage, etc.

Blocks compose with other blocks. Almost anything can be created from this, the lowest of layers. And when you create a new primitive that fits the interfaces, creatively can explode with the possibility.

A LEGO car built from a set

Sets

Starting from first principles, from the lowest level of blocks, can be overly complex. In practice, someone can build the next level of abstraction that solves a problem and can share it with others.

These are patterns, or recipes, or… Sets. With Lego, you most often see people buying sets with instructions on how to build a collection. I remember getting instructions with many ideas that I could reuse blocks to create with.

In software we often see this grouping of capabilities in various frameworks, that come with their own instructions on how to put things together. At other times you see templates, where you have a starting point of blocks to give you a strong leg up. Vercel does a great job of providing these, making it easy to start building on their platform.

A LEGO car kit, very custom purpose

Kits

There is a slightly different form of abstraction on top of blocks, and that is kits. These are meant to be for a particular purpose and are more restrictive that sets. The blocks and setup fit together much better (are less blocky!) and you wouldn’t take the parts of a kit and use them to make something different.

In software, an extreme case would be a proprietary language with components that only let’s you build extensions that look and feel like the platform they run on. A kit would make sense if you wanted to be very restrictive on extensibility, and you value making it as easy as possible to do certain things, and hard to break out.

You only really want to codify kits when you are very sure that they are very common and useful. If you have a scenario where many people will want to clone and tweak, you may be on to something with a kit.

There are many valid cases for kits, but it is also true that too often companies make the choice to over-invent. It is so very tempting to create a domain specific language, or even a custom programming language. But first, consider codifying constraints using languages and platforms that many developers have spent the time to learn already, where there is community, and answers, and where AI tools have something to have been trained on 😉

You can poke and make fun of English, in the same way that you can do so with JavaScript, but there is a reason it is still thriving whereas Esperanto isn’t.

Can you see the blocks, sets, and kits in your platforms? Are they well layered?

NOTE; My good friend and “one of the best platform engineers I know”, Dimitri Glazkov, has written about these layers from a slight different lens in his great piece: 4 layers.

ai-hints.json: how the ecosystem will help the bots

As I use LLMs to help me build software I keep running into situations where there is a missing piece and leverage point, that if injected will dramatically raise the quality of creation: subject expert turtles.

Triangle showing how app devs, experts, and bots come together

Much of this formed when creating mock.shop and seeing what it takes to go from a demo to production, but let me explain via another recent experience: a web app framework migration.

Framework migration: switching between Next.js and Remix

Next and Remix merging

LLMs are helpful for tasks such as porting. I have used this often, especially working between Python and JavaScript for some recent AI projects. I wanted to explore taking a web application using Next.js and have it ported to use Remix, or vice versa.

Out of the box, with ChatGPT, it would get some of the high level changes correct, but it would be very surface level. For example, actions and loaders may be created, but imports would have the form of @/components/foo and next/image.

Our LLM friend has a galaxy of information, but we don’t know what’s actually in there, and software keeps evolving so the information that may be there is probably outdated to some degree. This is where us humans come in. We can use that juicy context window to share:

  • The latest information from documentation that maps to our versions. Querying embeddings from this content can be plucked into context.
  • Rules and reasoning for translations. What are the steps that someone knowledgeable of both frameworks would write?
  • Quality examples of before and after. If you go through these steps what are solid mappings where patterns can be learned?

Depending on the quality of this work, you will see a massive upgrade in the results. They go from “some nice hints but wow so much is wrong” to “this is kinda usable out of the box!”

At the end of this process, “What are the steps that someone knowledgeable would write?” stuck with me. Someone else was going to go through the same migration, and it doesn’t make sense for them to have to build out all of the mappings. This is a waste of effort!

Time for the knowledgable turtles, already!

I have some knowledge of Next.js and Remix, but I am hardly The Expert. What if true experts (core team, folks from the community, etc) were the ones to package the relevant information about their frameworks?

Lee does great streams like this!

Lee does a great job showing a conversion from one version of next.js to another (to App Router land). This knowledge can be codified for anyone else doing this.

Picture an app developer creating a new project and installing all of their dependencies, and this time each one of them comes with hints from the projects themselves. And it’s turtles all the way down as each dependency comes with it’s own dependencies.

In this world you are building with a world of expertise funneling information into the amazing reasoning engine that is AI via LLMs.

What knowledge can we funnel?

Others are talking about this

Each project has an ai-hints.json file, which is the router to correct information. It is a simple configuration that links out, or contains some inline information, for the given project.

It contains items such as:

  • URL to the source of the library (e.g. GitHub URL)
  • URL to the home page of the library
  • Description for the library
  • URL to issue tracker of the library. Given the variability of quality in here, pinches of salt are included, and can map to answers from trusted folk / voted up
  • URL to forums (e.g. StackOverflow / tags)
  • URL to documentation site(s)
  • URL to high quality community content (e.g. great blogs, YouTube, etc)
    • Popular libraries often bring in examples, and other projects that use them, and run their test suites as a great way to catch regressions that your consumers will run into. We can follow this pattern to get wisdom from customers not just official content
  • Versioning scheme:
    • One current issue is that LLMs aren’t aware of the differences between versions and thus you sometimes get feedback that is tied to an old version, which is frustrating!
  • URL or direct inline prompts that can be used to generate great tests
  • URL, or inline docs, to prompts and reasoning
    • This can become a store of knowledge. E.g. it can be where conversion knowledge goes
  • URL to project settings such as package.json in node / js ecosystem to start to find all of the turtles
  • Polymath services: URL(s) to polymath services that have knowledge of the project
  • Embedding stores: URL to a store, or a local placement of embeddings that can be used and aggregated
    • This way we can share embeddings versus recreating them time and time again

Speaking your full language

There have been some moments where my AI pair has been a true partner. A pattern in most of the best moments has been how the back and forth can be so much more concrete. Often, if you are building something for your own application, you end up following a path of translation.

You want to do concrete thing X using library Y. This would often resort with finding the documentation in various places and learning the abstract thing closest to X (which may not be easy to even find!) and then working out how to translate this information into what could work for the concrete task.

Now, you can *explain* the concrete task, explain that you want to use it with your set of tools, and the initial answers can be speaking in that language. And you may not even know which library to use, and you can ask for thoughts and implementations with those thoughts too!

Being able to aggregate the dependencies is huge.

One of the reasons I enjoy working on Polymath is it’s federated nature. If I am working on a project that uses Remix with Preact I can write a query that asks for information from both the Remix polymath and the one for Preact.

More knowledge? More context

Billions of tokens!

As we build out larger knowledge sets, we need new ways to feed our AI’s creativity. Fortunately, we are seeing various models get significantly larger capacity for prompt tokens, including updates today from OpenAI:

“gpt-3.5-turbo-16k offers 4 times the context length of gpt-3.5-turbo at twice the price: $0.003 per 1K input tokens and $0.004 per 1K output tokens. 16k context means the model can now support ~20 pages of text in a single request.”

OpenAI announcement on June 13th 2023

We are also getting smarter with how we can chain reasoning together. Instead of firing off one prompt as a shot, you can do multiple, and ask various questions to drastically improve quality too.

E.g.

  • Ask questions differently in parallel
    • Use different prompts
    • With different settings (e.g. multiple temperature values)
    • With different context
    • And even different models entirely
  • Using the output from above, ask for a critique
  • Feed the critique AND the options from above, and ask for a unified solution.

Ecosystem scratches it’s own back

Large refactoring that works

By coming together and curating the information, we not only scratch the backs of all developers using our products, but in turn we are helping ecosystem health.

If you have worked on a platform, you know that one of the most important things to do is setup incentives to keep the platform evolving and healthy.

This is hard to do, and often goes wrong. You have probably worked with tools where updates break things more often than not. What does that teach you to do? Lock in to a particular version that is working, and only do upgrades when you have the time to deal with it.

If instead, upgrades work well? Then you should be game to continuously keep up to date. Doing well would look like:

  • Clear understanding of what’s in the upgrade
  • Codemods that can run to help you update. We will soon be in a world where our nano bots will see an update, create a PR, run all of our tests, and you will have a strong starting point, or maybe even more. I’m ready for my nanobots to be cleaning things up for me, handling updates, performance improvements, checking for security, for accessibility, etc etc.
    • We will just need great UX so we don’t feel swamped with this work. I don’t want a poor open source maintainer to be slammed with PRs in a way that it feels like spam. Categorization etc on GitHub will be a big winner here 🙂

An entirely revolution is happening when the world of developers and shared knowledge combines with the new world of LLMs.

We don’t want LLMs to use the overall corpus of code that is out there, because if the surface of the code is small, it may not have any great answers, and when the commons is huge, you can end up with a lowest common denominator such as:

The commons sometimes catches div-itis and worse

When it comes to code, hallucination is the enemy not just because it is very unhelpful, but also due to side effects such as security:

Humans write a lot of bad code too, so let’s be more vigilant for all?

It’s not just about the quality of the code and the ease of use. It’s also about the speed at which we can learn, adapt, and grow as developers. With a wealth of knowledge at our fingertips, we can quickly understand new technologies, make informed decisions about the tools we use, and stay ahead of the curve in an ever-evolving industry. This revolution in software development is empowering us to build better, more efficient, and more innovative applications than ever before.

I’m very much here for it, and Yet Another robots.txt^H^H^Hjson.

/fin

ps. In some time some of the expert turtles will become robot turtles 😉

We keep confusing efficacy for effectiveness

I often see the same debates occur, especially around web technology. One pattern I see repeated in these debates is how focus is often on the efficacy of a certain technical approach (or tool or library or…) and the practical effectiveness is ignored.

What the eff are you talking about?

Before we get into web performance and tech, what’s the actual difference between efficacy and effectiveness?

“Efficacy can be defined as the performance of an intervention under ideal and controlled circumstances, whereas effectiveness refers to its performance under ‘real-world’ conditions.”

A Primer on Effectiveness and Efficacy Trials

I remember when Peter Attia spoke about this in regards to his fish oil consumption. Is It better to take fish oil liquid, or fish oil tablets?

“The fish oil liquid is more potent and more concentrated in a higher dose, and if I take it every single day, I get higher levels in my red blood cell membranes (how we measure levels of EPA and DHA), which is the desired effect. But on average I forget to take it at least 2 times per week because the bottle needs to sit in the fridge and I forget to take it. Conversely, the fish oil capsules are easy for me to put in my pill pack next to my sink, which I never forget to take, but they are not as potent. So, while the liquid is more efficacious, the capsules may be more effective.”

— Peter Attia

Yup, we have all been there. What’s key here is both the fact that Peter actually measures the output (red blood cell membranes) and also the practical input. The details matter. How much is the difference between the two? Is the practical approach so weak as to require doing everything possible to make sure to suck it up and take the liquid? Good to go deeper, but our gut can also tell us that taking something is better than nothing.

You quickly see the efficacy vs. effectiveness pattern everywhere. Let’s bring it into our day jobs and how we debate the usage of technology and the outcomes at the other end.

The reason that benchmarks are so hard to do well, is that they measure a pure path that isn’t actually realistic at all. They are like running traffic directions algorithms without taking into account the other vehicles and traffic patterns on the road, or weather, or the time of the day, etc.

We are debating options while missing many many variables, and ones that really do matter. A subset of them are:

How easy is it to stay on a well lit path?

It is one thing for the creator of the technology to build something they have done before using their creation. But how is it for the average engineer? How variable are the outcomes based on the constraints of the system, the tooling, the documentation, the community, and on and on.

How does your app scale?

The performance of hello world is ~insignificant. How fast if your typical information, and what happens as you add features (code)? I would much develop with a system that allows me to add functionality that balances business value with performance. Compare this to something where the initial scaffolding is faster, but everyone slows over time.

What are you actually building?

The Web in particular is a general framework that has a large spectrum of use cases deployed on top of it (the spectrum mentioned here). The characteristics vary wildly as you look at the relative importance of initial load, amortizing costs for longer app sessions, requirements for collaboration, and more.

With Active Recall, which is an offline-first rich application, data is sync’d in a way that means you are basically always seeing optimistic UI operations. You are logged in by default, and the focus is on very fast actions while you use the application. These requirements land you with a very different architecture where the initial bundle isn’t the focus.

DX vs. UX

So, the next time you see another debate popup pitting DX and UX, remember that they aren’t binary variables at odds with each other. Developer productivity matters, can result in better UX, and make sure that you measuring effectiveness as it relates to your precise context.

And If you work on a platform, or on tooling, get real and do everything you can to help developers where they are, and don’t let perfection become the enemy of progress.

The right thing to do, is the right thing to do.