The non-existence of ACID consistency - Part 3

In the previous post of this blog series, we discussed why the “strong consistency is needed for business reasons” requirement is void. In this post, we will discuss the actual value of strong ACID-like consistency.

The real value of ACID consistency

The previous post may have left the question: Are strong consistency and ACID transactions pointless?

The definite answer to that question is: No, not at all!

Strong consistency does have a huge value. But its value does not lie in some business or user need that requires all data copies everywhere to be in sync all the time, that prohibits any non-atomic updates across system boundaries.

As we have discussed in the previous post, those kinds of demands cannot be justified with needs outside the IT systems. The world outside IT is eventually consistent at best. Thus, any consistency demands stronger than eventual consistency cannot be rooted in actual user needs.

The actual value of ACID transactions lies somewhere else. It lies in the much simpler mental model – especially for developers.

Just think about it for a moment: You have a series of dependent data changes. You need to execute all of them to move from one consistent state to the next consistent state. If you do not execute all of them, your data will exhibit inconsistencies. This is hard to reason about – and it is very hard to design and build your application in a way that it can deal with arbitrary kinds of inconsistencies.

This is the biggest value of ACID transactions: You start with a consistent state. You begin a transaction. You apply the changes inside the transaction. Either all changes are successful (commit) and you moved to the next consistent state. Or none of the changes are executed (rollback) and you end with the initial consistent state. Nothing between. No inconsistent state with just some of the changes applied. That is what the “A” in ACID means: Atomicity – all or nothing updates.

Additionally, from the outside you either see the old state or the new state. You do not see any potentially inconsistent work in progress that happens inside the transaction from the outside. That is what the “I” in ACID means: Isolation – interim inconsistent states while the transaction is in progress not becoming visible.

This is great stuff!

This makes reasoning about state, state transitions, concurrent data access and updates and all that so much easier!

ACID transactions are bliss for every software engineer.

Therefore, at least I do not want to miss strong consistency. I am a strong advocate of using it wherever it is applicable in a sensible way.

Giving up strong consistency for the wrong reasons

I never understood why developers in the NoSQL hype times so desperately wanted to throw their RDBMS out of the window and “replace” them with a NoSQL solution like MongoDB or Cassandra – just to mention the two most prominent NoSQL solutions of that time period.

Do not get me wrong: Most NoSQL databases, including MongoDB and Cassandra were and are great solutions for certain kinds of use cases and I am glad we have them as additional storage solution options. But they are not a “drop-in replacement” for a RDBMS. They are something completely different.

To stick with MongoDB and Cassandra as those were the usual choices in those days: Both do not provide ACID transactions out of the box. With their default settings, they do not even guarantee eventual consistency, let alone strong consistency. Both were tuned for availability – which is totally legit for the types of use cases they were originally designed for.

During the NoSQL hype, at best you could tune them up to eventual consistency using appropriate quorum settings – again, perfectly fine for the types of use cases they were originally designed for ¹.

Nevertheless, lots and lots of developers tried to squeeze these databases into every new project, trying to avoid the use of a RDBMS at all costs. ²

At development time this still felt good. But then the applications went live. And the databases went inconsistent in no time. Consistency hell broke loose.

The developers had their database accesses programmed as if they were talking to a relational database with ACID transactions taking care of consistency. But there were no ACID transactions. At best there were BASE transactions providing eventual consistency.

Most people back then did not think about consistency at all when trying to force these new databases into their projects. They were so used to the strong consistency guarantees of their RDBMS, they did not ponder what the absence of strong consistency would mean. They only knew a world where strong consistency was “natural”. Pondering the consequences of weaker consistency guarantees thus was beyond their imagination.

As a consequence, they typically used the new NoSQL databases “as is”, ending up with what I tend to call “accidental consistency” – no consistency guarantees at all beyond the scope of a single update of a single entity on a single machine.

“Accidental consistency” in practice means you will end up with persistent inconsistencies for sure. This is a situation you should avoid by all means. At least you should have eventual consistency in place, the weakest form of consistency which guarantees data consistency.

But eventual consistency still means you need to take potential temporary inconsistencies into account when you design your application logic and database accesses. You need to check for potential temporal inconsistencies and implement provisions to handle them in your application code.

This makes reasoning about state, state manipulation, (concurrent) data access and application logic based on what you see hard – really hard. Sometimes, it can be mind-bending.

In 2013, quite at the peak of the NoSQL hype, I gave a talk about the effects of switching from ACID to BASE transactions and what you need to do to compensate the effects. I still remember the faces in the audience going from relaxed to tense to anxious to horror and then flipping forth and back between the latter three states.

Most people never really pondered the consequences of switching from ACID to BASE consistency and many people are still shocked if you explain the effects to them ³.

That is why I recommend to stick to ACID transactions if there is not an important reason not to do so. Giving them up just because “you can” or “it’s cool” is a bad idea and you inevitably will pay a high price for it – either in development due to much harder reasoning about state or in production due to inevitable inconsistencies in your production databases.

Distributed systems create different forces

Still, in distributed systems prevail different forces. Strong consistency across process boundaries comes at a price. If you look at distributed data stores (no matter if we look at different databases or replicas of the same database), the CAP theorem tells us that we need to decide if we optimize for consistency or availability.

Optimizing for consistency, i.e., demanding strong consistency across multiple data nodes, means that you sacrifice availability for consistency. If anything should go wrong in the communication between the nodes involved, you will rather refuse to answer a read request than potentially letting different nodes answer differently. You will also rather abort updates than not updating all nodes involved at once.

This can be a legit strategy. But as written before: It compromises availability. You may not be able to write any updates for a while because one of the nodes involved is not able to process updates – which by the way from a functional perspective means your users see outdated data: Newer data is available and known to exist; the systems just cannot process the pending updates because update in lock-step is required.

Additionally, most systems today – at least in commercial contexts – have very high availability requirements. 99,9% availability 24/7 are quite normal. A scenario where (distributed) transactions fail regularly as described in the first post does not fit in such a setting.

In such a setting, it is mandatory to strive for eventual consistency. In such a setting, it is much more valuable if 90% of your nodes have the new data and deliver it than no nodes having the new data because a single node is down at the moment.

Just be aware that you need to go the extra mile to detect potential state deviations and respond to them at the application level.

Thus, my overall rule of thumb is:

Go for strong consistency on a single node.

Go for eventual consistency if multiple nodes are involved.

(And do not forget to handle potential temporary state inconsistencies at the application level.)

Of course there would be a lot more nuances to add. But as discussing these nuances in detail would rather result in a whole book than a blog post, I will leave it here.

Summing up

In this post, we discussed the actual value of strong ACID-like consistency which is that it makes reasoning about state, state transitions and (concurrent) data access and updates a lot easier.

In the next and final post of this little blog series, we will discuss why “ACID” is usually not what you think it is. Stay tuned … ;)