• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Dion Almaer

Software, Development, Products

  • @dalmaer
  • LinkedIn
  • Medium
  • RSS
  • Show Search
Hide Search

Scaling with Immutable Data; When Tech meets Retail

November 22, 2012 Leave a Comment


I am enjoying my second Black Friday in retail (I work for Walmart Labs Mobile, and we create apps for the global Walmart community). This is game time for us, and along with the nervous stress you get a lot of the joy. You want to be in the big game.

I am new to retail, so it has been a fun learning experience to see what it is all about, and I have been able to meet amazing retailers. I wanted to tell one simple story about thinking through a problem with a retailer.

Today is a run up to Black Friday in the US, and we have been releasing new deals throughout the day, a bunch of them exclusive deals via our apps (iPhone, iPad, Android).

If you are a retailer in the store, how may you role these out? Maybe you would have a special shelf at the front of the store, and for the first run you put out the Furby’s (until they quickly sell out). Then for the second run you put out the TVs, and then the Waffle Irons…. you get the picture. The problem though is that if items don’t sell out, then you may have created a bottleneck in that area of the store. For Black Friday in our stores for example we place the specials all over the place so customers can get some space (which is tough with the number of them in the store!).

Now, let’s move over to the online world. In the eCommerce platform we have a notion of a “shelf” too, which houses product. The natural solution would be to slowly add new products to the magic shelf as the day progresses, and then any items that are not sold off can hang out there. Great!

Well, maybe not so great when it comes to scaling up the solution. We have a crap ton (technical term) of online users hitting us at the same time. Caching is our friend, and the friend of caching is immutable data. It isn’t just for data sets in your code.

To help us out here, we actually setup multiple shelves for each event.

Shelf #1 @ 7am: Furby

Shelf #2 @ 9am: Furby and TVs

Shelf #3 @ 11am: Furby and TVs and Waffle Irons

And so on. Our apps and experiences have pointers to these shelves, so all we need to do is change the pointer, and the apps see the other shelf. The advantage here is that the apps never have to think about reloading a shelf, as the high level product data is the same (there are details within that matter such as product availability but the structure of shelf -> [ products ] is static).

It is a little more work for the merchants to setup as they need to duplicate the product info, and we thank the merchants for doing that work…. but it means that through the immutability we can scale much easier, and the clients can be so much simpler.

The moral of the story for me has been, how can we make as many of our data structures immutable, which often means thinking about the problem differently to how you may do so in a physical store.

After all, as a wise person said:

The basic goal of a database is to maintain the illusion that there is only one copy, only one person changes it at a time, everyone always sees the most current copy, and it is instantly fast. This is impossible to achieve.
This is especially impossible when millions of people are accessing and updating trillions of data elements from all over the world.
The task of database design, therefore, is to come as close to this illusion as possible using hundreds of interlocking algorithmic tricks.

Share this:

  • Twitter
  • Facebook

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

Twitter

My Tweets

Recent Posts

  • Generative AI: It’s Time to Get Into First Gear
  • Developer Docs + GenAI = ❤️
  • We keep confusing efficacy for effectiveness
  • The holy grail of a Web SDK
  • The rise of the extensible app platforms

Follow

  • LinkedIn
  • Medium
  • RSS
  • Twitter

Tags

3d Touch 2016 Active Recall Adaptive Design Agile Amazon Echo Android Android Development Apple Application Apps Artificial Intelligence Autocorrect blog Bots Brain Calendar Career Advice Cloud Computing Coding Cognitive Bias Commerce Communication Companies Conference Consciousness Cooking Cricket Cross Platform Deadline Delivery Design Desktop Developer Advocacy Developer Experience Developer Platform Developer Productivity Developer Relations Developers Developer Tools Development Distributed Teams Documentation DX Ecosystem Education Energy Engineering Engineering Mangement Entrepreneurship Exercise Family Fitness Founders Future GenAI Gender Equality Google Google Developer Google IO Habits Health HR JavaScript Jobs Jquery Kids Stories Kotlin Language Leadership Learning Lottery Machine Learning Management Messaging Metrics Micro Learning Microservices Microsoft Mobile Mobile App Development Mobile Apps Mobile Web Moving On NPM Open Source Organization Organization Design Pair Programming Paren Parenting Path Performance Platform Platform Thinking Politics Product Design Product Development Productivity Product Management Product Metrics Programming Progress Progressive Enhancement Progressive Web App Project Management Psychology Push Notifications pwa QA Rails React Reactive Remix Remote Working Resilience Ruby on Rails Screentime Self Improvement Service Worker Sharing Economy Shipping Shopify Short Story Silicon Valley Slack Software Software Development Spaced Repetition Speaking Startup Steve Jobs Study Teaching Team Building Tech Tech Ecosystems Technical Writing Technology Tools Transportation TV Series Twitter Typescript Uber UI Unknown User Experience User Testing UX vitals Voice Walmart Web Web Components Web Development Web Extensions Web Frameworks Web Performance Web Platform WWDC Yarn

Subscribe via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Archives

  • January 2023
  • September 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • November 2021
  • August 2021
  • July 2021
  • February 2021
  • January 2021
  • May 2020
  • April 2020
  • October 2019
  • August 2019
  • July 2019
  • June 2019
  • April 2019
  • March 2019
  • January 2019
  • October 2018
  • August 2018
  • July 2018
  • May 2018
  • February 2018
  • December 2017
  • November 2017
  • September 2017
  • August 2017
  • July 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012

Search

Subscribe

RSS feed RSS - Posts

The right thing to do, is the right thing to do.

The right thing to do, is the right thing to do.

Dion Almaer

Copyright © 2023 · Log in