Cloud native database

Managing a traditional database in the cloud is hard and risky. As we will discuss, it is necessary to replicate the data across availability zones to ensure the data is accessible and durable, and replicate it across regions to protect against regional outages. Managing a modern sharded database in the cloud is superhard and risky. We have all the complexity of managing the traditional database across availability zones and regions multiplied by the number of shards. Managing N modern, sharded databases in the cloud, to gain the advantages of Polyglot Persistence and bulkheads between components, is arguably an exercise in insanity, even for many unicorn companies. Imagine this oversimplified example: you have five components, each with two types of storage, each with four shards, deployed across two regions, with three availability zones each. So that is 5 * 2 * 4 * 2 * 3 = 240 elastic block storage devices that must be provisioned, monitored, and managed. Obviously, this gets expensive pretty quickly, but that is not the real concern. With these numbers, and this is an oversimplified example, the probability that a storage device fails on any given day is very high. Of course, the point of this approach is to keep the database available during these failures, but the point here is that this is a maintenance nightmare. Does this mean we throw out the approach and go back to our shared monolithic database with no bulkheads and cross our fingers? No, because crossing your fingers won't help. That big outage will happen, it is only a matter of time, and it will happen at the most inopportune time, and the aftermath won't be pretty.

The alternative is to leverage value-added cloud services, as we discussed in the first chapter. We achieve more by managing less. However, any ole database-as-a-service offering is not what we are talking about. Those still tend towards a monolithic, shared database model. The best alternatives are the cloud-native databases. These databases are purchased, provisioned, and tuned one table at a time. This means that one table can be a document store, another table is a search index, and another table is blob storage, and so forth. They all operate across availability zones as a matter of course and more and more of these offerings have turnkey regional replication. Storage space for most offerings grows automatically, otherwise, storage space monitoring is provided. Read and write capacity will also auto scale. This is important at both extremes because it also means that they are a good alternative for small workloads as well. Many of these cloud-native databases are actually designed to support this inside-out database architecture and expose their transaction logs as an event stream, to facilitate the Event Sourcing and CQRS patterns. We will dive deeper into this in the Cloud Native Databases Per Component pattern section in Chapter 3, Foundation Patterns.

I think it should be starting to become clear that rewiring the way we think about systems, combined with leveraging value-added cloud services, can empower everyday companies to create reactive, cloud-native systems that are responsive, resilient, and globally elastic. It was a major wow moment for me when I realized that the Cloud is the database and how much that means we can accomplish with the fundamental cloud-native building blocks of event streaming and cloud-native databases.