1. CAP THEOREM Have the rules really changed?

2. Tolerance to Network Partition
Consistency
Availability
Tolerance to Network Partition
Basics of CAP Theorem

3. Basics of CAP
Any distributed system can satisfy any two of the three properties at the same time.
Consistency - Having single up-to-date copy of the data at the same time with all the nodes
Availability - The system would continue to operate as expected even with node failures
Partition-tolerance: The system would continue to operate as expected despite network or message failures

4. Inconsistent Server 1 Server 2 Server 1 Server 2 Server 1 Server 2x x y x y x y x y x
write
written
read x z z z z z
Client
Client
Client
Client
Client

5. Availability Client, Server Example Server 1 Server 2
Client sends a request for an operation to the server
The server must eventually respond to the client z
Client

6. Partition Tolerance Client, Server Example Server 1 Server 2
This system is not partition tolerant as S1 is updated S2 z
Client

7. WHY is it important??
A proper understanding of CAP theorem is essential to making decisions about the future of distributed database design
Misunderstanding can lead to erroneous or inappropriate design choices
The notion of having at most of “2 of these 3” is no longer strict as before
Choosing to have either Availability or Consistency is difficult

8. CAP - A popular misconception
The CAP categorizations - AP, CP, CA say very little about the actual behaviour of the distributed system.
How about CA?
Can a distributed system really be not tolerant of partitions?

9. CAP is not binary Out of the three, none of the properties are binary
Availability can be scaled from zero to hundred percent
Consistency also has many variances between being consistent and inconsistent
Though not well investigated, but network partitioning also has different degrees
CAP is not binary

10. Facebook Example
Suppose your friend finds something interesting on Facebook and shares it on your wall asking, you to check it out.
Where is it?
But when you login into your account you see nothing.

11. BUT WHY?? Facebook Example Contd.
Eventually after waiting for a minute, when you go back and try to check it out again, you find the story.
The actual reason – Facebook uses an eventual consistent model over the strong consistent.
BUT WHY??
It is non-trivial for them to store the huge amount of data collected/generated, efficiently and reliably as they have billion of users.
Using eventual consistent model offers to reduce the load and in turn improve availability.

12. How to Decide?

13. CAP confusion and Partition Decision
Contrary to what people believe or understand, CAP is not a one time decision.
No need to choose between consistency or availability beforehand.
A system can guarantee both availability and consistency when fully connected.
This decision can be left to answer up until the moment when the partition happens.
It can be changed multiple times based on different actions and data.

14. 1 2 Different Options for Partition Decision
Decrease availability by cancelling the operation 1
Risk inconsistency and proceed with the operation 2

15. Partition Recovery

16. Partition and its effects
At some point, communication resumes and the partition ends.
A log (history) is being is being kept during the partition mode.
After the partition, system knows the state and history of both sides.
System can deduce operations that actually violated invariants.

17. The state on both sides of the partition must be made consistent
Partition Recovery
When the partition finally ends, a system is required to perform a so called Partition Recovery Step.
Two hard problems during recovery are:
The state on both sides of the partition must be made consistent 1
Undo any mistakes 2

18. More on Partition Recovery
It is easier to fix by starting from the state at the time of partition.
Rolling forward with both sets of operations is the way forward.
Some systems can merge conflicts but some can’t.
Designers can choose to avoid the use of certain operations at the time of partition.
System can automatically merge during recovery.

19. Achieving Partition Recovery
Using CRDTs - commutative replicated data types 1
Using compensating transactions 2

20. CRDTs
CRDTs - class of data structures that help in maintaining commutative operations and data merging.
If operations on your data types are both commutative and idempotent
Nodes in a distributed system are able to arbitrary leave and rejoin a network
They can send duplicate messages at any point, and
Maintain eventual consistency in the whole process.

21. CRDTs Continued
It doesn’t matter with what order the operations on the data types are performed.
Even if you perform an operation multiple times it has no effect on the state
In the end there is a guarantee that all the nodes achieve the same state after applying all pending operations.

22. Compensating Transactions
Second step towards Partition Recovery
Fixing any inconsistencies after a partition is resolved
Mistakes made have to be corrected
So-called externalized operations (actions that let out information to other systems or users that are not in control of the original system) need to be analyzed
Checked for errors and compensated.

23. Compensating Transactions
Various ways to fix invariants -
Last writer wins – not the smartest, as ignores some updates
Approach that can merge operations
Human escalation

24. Example - Automatic Merging
Airplane overbooking
Airplane booking system allowing booking of seats in partition mode
Would work out if the flight is not full
If there are too many customers, some would loose their seats
Having the history makes it easier to resolve the issue
Customer Service can compensate in some way

25. Brewer’s conclusion
System designers should be aware of their system closely
Look at each invariant and design the system to react on the network partition
He sums it up very concisely –
“System Designers should not blindly sacrifice consistency or availability [...].
They can optimize both properties through careful management of invariants during partitions.”

26. My conclusion
The publication is interesting specially coming from the inventor of CAP theorem himself
Article is very good for newbies learning about the basics of CAP theorem
Manages to bring in interesting thoughts and mindsets for more experienced developers
Things I liked:
Lays down great emphasis on the fact that CAP allows for very fine grained decisions
Opposes my initial understanding of it being a binary formulation of the theorem

27. Shortcomings Not any groundbreaking new theory
Just a summation of possibilities and technologies that have already been researched over the last decade
Everything boils down to a unified model

28. Thank You!