ten billion hits a day?

I saw Chander Dhall give a talk called "Best practices for scaling web apps (Platform Agnostic)" at the "Rockstar Developers/Designers Meetup" on Thursday night. The talk started out with theoretic "whiteboardings" in a slide presentation about how an architecture could start for a web application and how attempts are typically made to improve the architecture when it is not enough. Logical abstractions were collected together in dotted-lined circles and rectangles while straight lines denoted physical abstractions such as the app itself and the database starting out running at two different servers in the simplest example before any augmentations/enhancements. That is how we all start isn't it? Going through a list of debunked but once industry-embraced solutions for scaling, Chander eventually began to explain sticky sessions and at this point a member of the audience challenged that he didn't think there was anything bad about sticky sessions and that they were solving his company's scaling woes just fine with millions of hits a day being handled. Chander suggested the need to accommodate millions of hits a day was really a smaller problem than he aimed to tackle in his presentation and that he had his sights set on the biggest of challenges, handling ten billion hits a day in an ecommerce environment. In such a circumstance, yes, sticky sessions are just not enough. The real crème de la crème challenge (the one that really says: "If you can do this you can do anything.") lies in online sales at venues like amazon.com, and herein Chander suggests that one should set one's sights on handling ten billion hits a day, a number, yes, higher than the seven billion that is the population of Earth. The social media space may get the same volumes of traffic, but these guys handle the challenge by caving into it a bit and they can get away with that where online realtors cannot. If we attempt to post to Facebook and something screws up and we have to just try again, well, it's considered acceptable. We are all just numb to that now aren't we? It is not alright to lose sales here and there online. Beyond requests just failing, sales may be lost in lag. Chander sighted many metrics to point to how sales volume decays the slower a site may run, the longer it takes to load the home page, etc. One percent of sales fell off from Amazon when their process ended up taking a tenth of a second longer. There is big money at stake in performance. So let's jump right to the end and see Chander's solution. It involves using three different data solutions at three different pages of a web site at the first three different steps working inward in process. Why not use MSSQL everywhere? Well, joins are expensive. There are faster ways to go than using relational data. And so, three steps:

1. The entry point should be assumed to be the home page of the site. While some may bookmark locales within ecommerce platforms it should be understood that most webfaring folk will just enter at the home page (the easiest URL to type and remember) and use the search box at the top of the navigation. Chander mentioned that he has never seen a home page hydrated by just one object, and indeed, beyond search, there is likely to be featured products on display at the home page of whatever.com and perhaps the menu system is also database driven. The important thing to keep in mind here is the search trumps all other objects in significance and you will want to get the search loaded and accessible first before you start pulling in the other stuff. Chander didn't explain how to do this, but I guess I'd pull in all secondary concerns by way of AJAX calls with JavaScript tags tucked away just before the closing body tag in HTML. This piece should be facilitated by Elasticsearch or Solr/Lucene. There will be an RDBMS (relational database management system) keeping records for the web site, yes, and to keep it jiving with this piece data we will have to be publishing data over, perhaps in a nightly process. At framesdirect.com this was being done, nightly, to facility Solr searching when I worked there.
2. Every page one step in from the home page, including, for example, the "About Us" and "Contact Us" pages, but more importantly, the landing pages one finds from searching with products in exposition with pictures and descriptions are driven by NoSQL solutions such as MongoDB and RavenDB, but Chander did not have nice things to say about MongoDB and RavenDB and instead encouraged the use of Cassandra. He didn't spend too much time on why Cassandra, but he did suggest that Mongo drops records. Going back to my framesdirect.com experience, at this point we would be seeing data hydrated from MSSQL (not this middle step) and I wondered aloud how inventory data (i.e. is the product you are looking at actually available for purchase?) could make it up to these pages if the NoSQL data was also, I assume, pushed into place by a periodic process from the RDBMS. Chander suggested that inventory concerns should be addressed at the moment of checkout and not at the moment things are added to a shopping cart. He suggested that his wife will often spool things up in her shopping cart at amazon.com in a wishlist manner and wait days before checking out, so in this scenario the inventory would have to be rechecked at purchase time anyways and the possibility that something was not in stock would have to be addressed then too. If you think about it, the discrepancy between we-have-it-now and we-don't-have-anymore can happen even in the one second fast-as-possible trip between adding to cart and the checkout processing too, especially so at a web site with ten billion hits a day or perhaps in a scenario when tickets are being sold to a rock concert everyone is clamoring to see. You have to have the we-don't-have-anymore logic downstream from here.
3. Next when we add stuff to a shopping cart we are in the RDBMS data's space. All of the pages to come downstream from here are driven by relational data. The real chokepoints have been addressed before these destinations.

Chander's final architecture looks like what is shown above. There were a lot more new (for me) things I learned about in this talk. Expect further blog postings on this subject matter.