Your CIO is all fired up about moving your legacy inventory management app to the Cloud. Lower capital costs! Dynamic provisioning! Outsourced infrastructure! So you get out your shoehorn, provision some storage and virtual machine instances, and forklift the whole mess into the stratosphere. (OK, there’s more to it than that, but bear with me.)

Everything seems to work at first. But then the real test comes: the Holiday season, when you do most of your online business. You breathe a sigh of relief as your Cloud provider seamlessly scales up to meet the spikes in demand. But then your boss calls, irate. Turns out customers are swamping the call center with complaints of failed transactions.

You frantically dive into the log files and diagnostic reports to see what the problem is. Apparently, the database has not been keeping an accurate count of your inventory—which is pretty much what an inventory management system is all about. You check the SQL, and you can’t find the problem. Now you’re really beginning to sweat.

You dig deeper, and you find the database is frequently in an inconsistent state. When the app processes orders, it decrements the product count. When the count for a product drops to zero, it’s supposed to show customers that you’ve run out. But sometimes, the count is off. Not always, and not for every product. And the problem only seems to occur in the afternoons, when you normally experience your heaviest transaction volume.

The Problem: Consistency in the Cloud

The problem is that while it may appear that your database is running in a single storage partition, in reality the Cloud provider is provisioning multiple physical partitions as needed to provide elastic capacity. But when you look at the fine print in your contract with the Cloud provider, you realize they offer eventual consistency, not immediate consistency. In other words, your data may be inconsistent for short periods of time, especially when your app is experiencing peak load. It may only be a matter of seconds for the issue to resolve, but in the meantime, customers are placing orders for products that aren’t available. You’re charging their credit cards and all they get for their money is an error page.

From the perspective of the Cloud provider, however, nothing is broken. Eventual consistency is inherent to the nature of Cloud computing, a principle we call the CAP Theorem: no distributed computing system can guarantee (immediate) consistency, availability, and partition tolerance at the same time. You can get any two of these, but not all three at once.

Of these three characteristics, partition tolerance is the least familiar. In essence, a distributed system is partition tolerant when it will continue working even in the case of a partial network failure. In other words, bits and pieces of the system can fail or otherwise stop communicating with the other bits and pieces, and the overall system will continue to function.

With on-premise distributed computing, we’re not particularly interested in partition tolerance: transactional environments run in a single partition. If we want ACID transactionality (atomic, consistent, isolated, and durable transactions), then we should stick with a partition intolerant approach like a two-phase commit infrastructure. In essence, ACID implies that a transaction runs in a single partition.

But in the Cloud, we require partition tolerance, because the Cloud provider is willing to allow that each physical instance cannot necessarily communicate with every other physical instance at all times, and each physical instance may go down unpredictably. And if your underlying physical instances aren’t communicating or working properly, then you have either an availability or a consistency issue. But since the Cloud is architected for high availability, consistency will necessarily suffer.

The Solution: Rethink Your Priorities

The kneejerk reaction might be that since consistency is nonnegotiable, we need to force the Cloud providers to give up partition tolerance. But in reality, that’s entirely the wrong way to think about the problem. Instead, we must rethink our priorities.

As any data specialist will tell you, there are always performance vs. flexibility tradeoffs in the world of data. Every generation of technology suffers from this tradeoff, and the Cloud is no different. What is different about the Cloud is that we want virtualization-based elasticity—which requires partition tolerance.

If we want ACID transactionality then we should stick with an on-premise partition intolerant approach. But in the Cloud, ACID is the wrong priority. We need a different way of thinking about consistency and reliability. Instead of ACID, we need BASE (catchy, eh?)

BASE stands for Basic Availability (supports partial failures without leading to a total system failure), Soft-state (any change in state must be maintained through periodic refreshment), and Eventual consistency (the data will be consistent after a set amount of time passes since an update). BASE has been around for several years and actually predates the notion of Cloud computing; in fact, it underlies the telco world’s notion of “best effort” reliability that applies to the mobile phone infrastructure. But today, understanding the principles of BASE is essential to understanding how to architect applications for the Cloud.

Thinking in a BASE Way

Let’s put the BASE principles in simple terms.

Basic availability: stuff happens. We’re using commodity hardware in the Cloud. We’re expecting and planning for failure. But hey, we’ve got it covered.

Soft state: the squeaky wheel gets the grease. If you don’t keep telling me where you are or what you’re doing, I’ll assume you’re not there anymore or you’re done doing whatever it is you were doing. So if any part of the infrastructure crashes and reboots, it can bootstrap itself without any worries about it being in the wrong state.

Eventual consistency: It’s OK to use stale data some of the time. It’ll all come clean eventually. Accountants have followed this principle since Babylonian times. It’s called “closing the books.”

So, how would you address your inventory app following BASE best effort principles? First, assume that any product quantity is approximate. If the quantity isn’t near zero you don’t have much of a problem. If it is near zero, set the proper expectation with the customer. Don’t charge their credit card in a synchronous fashion. Instead, let them know that their purchase has probably completed successfully. Once the dust settles, let them know if they got the item or not.

Of course, this inventory example is an oversimplification, and every situation is different. The bottom line is that you can’t expect the same kind of transactionality in the Cloud as you could in a partition intolerant on-premise environment. If you erroneously assume that you can move your app to the Cloud without reworking how it handles transactionality, then you are in for an unpleasant surprise. On the other hand, rearchitecting your app for the Cloud will improve it overall.

The ZapThink Take

Intermittently stale data? Unpredictable counts? States that expire? Your computer science profs must be rolling around in their graves. That’s no way to write a computer program! Data are data, counts are counts, and states are states! How could anything work properly if we get all loosey-goosey about such basics?

Welcome to the twenty-first century, folks. Bank account balances, search engine results, instant messaging buddy lists—if you think about it, all of these everyday elements of our wired lives follow BASE principles in one way or another.

And now we have Cloud computing, where we’re bundling together several different modern distributed computing trends into one neat package. But if we mistake the Cloud for being nothing more than a collection of existing trends then we’re likely to fall into the “horseless carriage” trap, where we fail to recognize what’s special about the Cloud.

The Cloud is much more than a virtual server in the sky. You can’t simply migrate an existing app into the Cloud and expect it to work properly, let alone take advantage of the power of the Cloud. Instead, application migration and application modernization necessarily go hand in hand, and architecting your app for the Cloud is more important than ever.

 

Discussion
71 comments for “BASE Jumping in the Cloud: Rethinking Data Consistency”
How to leverage Atomikos in the cloud « blog.atomikos.com Avatar

[...] consistency in the cloud is a totally different ball game - as perfectly illustrated by this newsletter issue from ZapThink. Essentially, you have to cope with some form of partition tolerance and stale data values in [...]

RESTful Services: Key to Cloud Computing | Kavis Technology Consulting Avatar

[...] about data consistency. Here are two really good articles that dive deeper into this topic (from ZapThink  and HighScalability.com). ACID (Atomicity, Consistency, Isolation, Durability) transactions are [...]

life insurance claim lawyer attorney law firm Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Kenneth Oshiro Avatar

... [Trackback]

[...] Read More: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Kenneth Oshiro | July 27, 2016
Calile Malouf Investments Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Calile Malouf Investments | August 1, 2016
aromadex Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by aromadex | August 8, 2016
taruhan bola Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by taruhan bola | August 12, 2016
ayam petarung Avatar

... [Trackback]

[...] Read More: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by ayam petarung | August 12, 2016
Buy Steroids Online Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Buy Steroids Online | August 13, 2016
satta matka Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by satta matka | August 22, 2016
Joseph de Saram#Rhodium Avatar

... [Trackback]

[...] Find More Informations here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Joseph de Saram#Rhodium | August 26, 2016
Agen Bandarq Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Agen Bandarq | August 31, 2016
قاب گوشی Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by قاب گوشی | September 1, 2016
economics tuition Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by economics tuition | September 3, 2016
economics tuition Avatar

... [Trackback]

[...] Read More: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by economics tuition | September 3, 2016
yandex ankara Avatar

... [Trackback]

[...] Informations on that Topic: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by yandex ankara | September 4, 2016
yandex ankara Avatar

... [Trackback]

[...] There you will find 60356 more Infos: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by yandex ankara | September 4, 2016
yandex ankara Avatar

... [Trackback]

[...] Read More: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by yandex ankara | September 4, 2016
What is the definition of Marketing Plan at Black's Law Dictionary Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

obat herbal kutil kelamin wanita Avatar

... [Trackback]

[...] Read More: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by obat herbal kutil kelamin wanita | September 17, 2016
مقاله طراحی سایت Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by مقاله طراحی سایت | September 17, 2016
de nature herbal Avatar

Makanan yang sangat disarankan untuk ibu hamil

hi, my name is eko.. i like your website

Posted by de nature herbal | September 23, 2016
de nature yogyakarta Avatar

dapatkan pengobatan mujarab dari de Nature Indonesia

hi, my name is eko.. i like your website

Posted by de nature yogyakarta | September 23, 2016
Kulit Pisang Obat Kutil Kelamin Avatar

dapatkan pengobatan mujarab dari de Nature Indonesia

hi, my name is eko.. i like your website

Posted by Kulit Pisang Obat Kutil Kelamin | September 25, 2016
obat kutil kelamin Avatar

dapatkan pengobatan mujarab dari de Nature Indonesia

hi, my name is eko.. i like your website

Posted by obat kutil kelamin | September 26, 2016
Obat Sipilis Herbal dan Kelamin Bernanah Avatar

dapatkan pengobatan mujarab dari de Nature Indonesia

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Obat Sipilis Herbal dan Kelamin Bernanah | September 26, 2016
Kemaluan Keluar Cairan Seperti Nanah Avatar

dapatkan pengobatan manjur dari de Nature Indonesia

denature adalah sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Kemaluan Keluar Cairan Seperti Nanah | September 28, 2016
Kemaluan Laki Keluar Nanah Avatar

dapatkan pengobatan manjur dari de Nature Indonesia

denature adalah sebuah perusahaan yang bergerak dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Kemaluan Laki Keluar Nanah | September 28, 2016
Keluar Nanah Pada Kemaluan Lelaki Avatar

Keluar Nanah Pada Kemaluan Lelaki

denature adalah sebuah perusahaan yang bergerak dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Keluar Nanah Pada Kemaluan Lelaki | September 28, 2016
Obat Kemaluan Keluar Nanah Avatar

Obat Kemaluan Keluar Nanah

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Obat Kemaluan Keluar Nanah | September 29, 2016
digital marketing companies in Melbourne Avatar

... [Trackback]

[...] There you will find 73942 more Infos: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by digital marketing companies in Melbourne | September 29, 2016
Cara Mengatasi Lubang Penis Lecet Avatar

Penis Bengkak Gatal Gatal Kepala Pusing

Jual Obat deNature Herbal Ampuh Sembuhkan Kutil Kelamin Dalam 3 Sampe 5 hari.

Posted by Cara Mengatasi Lubang Penis Lecet | September 30, 2016
Motogolf Avatar

... [Trackback]

[...] Find More Informations here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Motogolf | October 2, 2016
Jual Obat Sipilis Ampuh Avatar

Jual Obat Sipilis Ampuh

de Nature Indonesia merupakan sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by Jual Obat Sipilis Ampuh | October 13, 2016
Can't wait to see what's next at Imprimiss Avatar

... [Trackback]

[...] Read More Infos here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Can't wait to see what's next at Imprimiss | October 14, 2016
Biaya Operasi Kutil Kelamin Avatar

Kemaluan Keluar Nanah

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Biaya Operasi Kutil Kelamin | October 14, 2016
wedang ronde Avatar

Keluar Nanah Dari Kemaluan

de Nature Indonesia merupakan sebuah CV yg bergerak dalam bidang pengobatan alami aman

Posted by wedang ronde | October 20, 2016
obat kencing Avatar

toko jual obat sipilis

de Nature Indonesia merupakan sebuah CV yg bergerak dalam bidang pengobatan alami aman

Posted by obat kencing | October 20, 2016
Best Newspaper in India Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Best Newspaper in India | October 21, 2016
kencing nanah itu apa Avatar

obat kencing

de Nature Indonesia merupakan sebuah CV yg bergerak dalam bidang pengobatan alami aman

Posted by kencing nanah itu apa | October 23, 2016
slot machines Avatar

... [Trackback]

[...] Read More here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by slot machines | October 24, 2016
rhodium Avatar

... [Trackback]

[...] There you will find 31411 more Infos: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by rhodium | October 31, 2016
Kenapa vagina Keluar cairan Putih Avatar

Kenapa vagina Keluar cairan Putih

Info Terkini Tetang Obat Herbal Denature Yang Terbukti Herbal Aman Tanpa efek Samping

Posted by Kenapa vagina Keluar cairan Putih | October 31, 2016
i love my financial systems plc Avatar

... [Trackback]

[...] Find More Informations here: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by i love my financial systems plc | October 31, 2016
Penis Keluar Cairan Warna Hijau Tiap Bangun Tidur Avatar

Penis Keluar Cairan Warna Hijau Tiap Bangun Tidur

Info Terkini Tetang Obat Herbal Denature Yang Terbukti Herbal Aman Tanpa efek Samping Hub 087802956210

Obat Lecet Pada Bibir Vagina Avatar

Obat Lecet Pada Bibir Vagina

Pengobatan Herbal Alami Tanpa efek Samping Dengan Obat Herbal Dari Denature

Posted by Obat Lecet Pada Bibir Vagina | October 31, 2016
http://bit.ly/2evJOTr Avatar

Kenapa Kemaluan Pria Keluar Nanah

denature adalah sebuah perusahaan yang bergerak dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by http://bit.ly/2evJOTr | November 1, 2016
cara Avatar

obat

de Nature ialah sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by cara | November 1, 2016
Rasa gatal di bagian kemaluan batang dan kepala penis Avatar

Obat Lecet Pada Bibir Vagina

Pengobatan Herbal Alami Tanpa efek Samping Dengan Obat Herbal Dari Denature

Best Newspaper in India Avatar

... [Trackback]

[...] There you will find 42100 more Infos: zapthink.com/2011/06/01/base-jumping-in-the-cloud-rethinking-data-consistency/ [...]

Posted by Best Newspaper in India | November 2, 2016
yah Avatar

apa

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping.

Posted by yah | November 5, 2016
http://bit.ly/2ef53fP Avatar

Keluar Nanah Pada Kemaluan Pria

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by http://bit.ly/2ef53fP | November 8, 2016
Obat Kencing Panas Avatar

Obat Kencing Panas

Obat Herbal Denatur Indonesia Yang Sudah Terbukti KeAmpuhanya

Posted by Obat Kencing Panas | November 14, 2016
Obat Kencing Perih Dan Keluar Nanah Pada Pria Avatar

Obat Kencing Perih Dan Keluar Nanah Pada Pria

Obat Kencing Perih Dan Keluar Nanah Pada Pria Herbal Denatur Indonesia Yang Sudah Terbukti KeAmpuhanya

Posted by Obat Kencing Perih Dan Keluar Nanah Pada Pria | November 15, 2016
cara mengobati ambeien tanpa operasi Avatar

cara mengobati ambeien tanpa operasi

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping.

Posted by cara mengobati ambeien tanpa operasi | November 16, 2016
Obat Gatal Pada batang Penis Avatar

Obat Gatal Pada batang Penis

Obat Gatal Pada batang Penis Herbal Denatur Indonesia Yang Sudah Terbukti KeAmpuhanya

Posted by Obat Gatal Pada batang Penis | November 17, 2016
nanah Avatar

nanah

denature adalah sebuah perusahaan yang bergerak dalam bidang obat herbal aman dan ampuh tanpa efek samping.

Posted by nanah | November 18, 2016
Penyebab Luka Lecet dan Gatal Pada Vagina Avatar

Penyebab Luka Lecet dan Gatal Pada Vagina

Penyebab Luka Lecet dan Gatal Pada Vagina Dan Pengobatanya

Posted by Penyebab Luka Lecet dan Gatal Pada Vagina | November 21, 2016
virus Avatar

virus

de Nature Indonesia merupakan sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by virus | November 22, 2016
pengobatan jengger ayam pada wanita Avatar

cara menyembuhkan kutil pada kelamin pria

de Nature ialah sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by pengobatan jengger ayam pada wanita | November 24, 2016
obat kutil kelamin malaysia Avatar

Obat Kutil Kelamin Malaysia

denature adalah sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping. obat kami sangat mujarab.

Posted by obat kutil kelamin malaysia | November 25, 2016
Lecet Pedih Dan Gatal Di Bibir Miss V Avatar

Lecet Pedih Dan Gatal Di Bibir Miss V

Lecet Pedih Dan Gatal Di Bibir Miss V Itu Gejala Penyakit Apa Yahhh

Posted by Lecet Pedih Dan Gatal Di Bibir Miss V | November 25, 2016
penyebab kutil di alat kelamin pria Avatar

Penyebab Kutil Di Alat Kelamin Pria

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping. obat kami sangat mujarab.

Posted by penyebab kutil di alat kelamin pria | November 25, 2016
obat oles kondiloma Avatar

Obat Oles Kondiloma

de Nature ialah sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by obat oles kondiloma | November 26, 2016
Obat Herbal Denature Online Shop Avatar

Obat Herbal Denature Online Shop

Konsultasikan Penyakit Anda Bersama Kami : Xl : 087802956210 M3 : 085747022427 Simpati :081296916270 Pin : D425DC73

Posted by Obat Herbal Denature Online Shop | November 29, 2016
Kemaluan Keluar Nanah Avatar

Keluar Nanah Pada Kemaluan Lelaki

denature adalah sebuah perusahaan yang bergerak dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by Kemaluan Keluar Nanah | December 4, 2016
obat kemaluan bernanah Avatar

Arti Mimpi Kemaluan Keluar Nanah

de Nature Indonesia merupakan sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by obat kemaluan bernanah | December 5, 2016
Obat Herpes De Nature Avatar

Obat Herpes De Nature

Konsultasikan Penyakit Anda Bersama Kami : Xl : 087802956210 M3 : 085747022427 Simpati :081296916270 Pin : D425DC73

Posted by Obat Herpes De Nature | December 7, 2016
de nature indonesia Avatar

Cara Mengobati Kemaluan Keluar Nanah

de Nature Indonesia merupakan sebuah CV yg berjalan dalam bidang pengobatan alami aman

Posted by de nature indonesia | December 7, 2016
obat kutil kemaluan Avatar

Kemaluan Pria Keluar Nanah

denature Indonesia yaitu sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by obat kutil kemaluan | December 8, 2016
http://bit.ly/2h8KGC1 Avatar

Mengobati Kemaluan Keluar Nanah

denature adalah sebuah perusahaan yang fokus dalam bidang obat herbal aman dan ampuh tanpa efek samping

Posted by http://bit.ly/2h8KGC1 | December 9, 2016