1. Joint Advanced Students School 2005
Replication and synchronization techniques in the context of mobile applications
By Alexander Stage
Software Engineering betrieblicher Informationssysteme (sebis) Ernst Denert-Stiftungslehrstuhl Lehrstuhl für Informatik 19 Institut für Informatik TU München
wwwmatthes.in.tum.de
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 1

2. Agenda Motivation Mobile Computing Synchronization Techniques
Replication Techniques in Databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 2

3. Motivation – A Scenario
The disconnected sales-person problem: a typical day
Mobile Device
Local DB
synchronize
Mobile Device
Local DB
replicate
Company
Central DB
Mobile Device
Local DB
update
no communication link
travelling path
Customer A
Customer B
Customer C
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 3

4. Agenda Motivation Mobile Computing Synchronization Techniques
Replication Techniques in Databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 4

5. Mobile Computing - Characteristics
Mobile devices:
Laptops
Palmtops
Smart cell phones
Requirements:
Data access:
Anywhere
Anytime
Nomadic users
Constraints:
Limited ressources
Variable connectivty:
Performance
Reliability
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 5

6. Mobile Computing – Communication Models
Client Server Model
Peer to Peer Model
Client
Client
Client/
Server
Client/
Server
Server
Client/
Server
Client/
Server
Client
Client
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 6

7. Two Types of Synchronization
Process synchronization
Data synchronization
System A
System B
Process A
Process B
modification
Process
Process
Time
Data item 1
modification
modification
Synchronization
Data Item 1
Data Item 1
Process A
correspondance
modification
Data item 1
Time
1: 1 Synchronization
1: 1 Synchronization
Main System
Process B
modification
Data Item 1
Data item 1
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 7

8. Aspects of Synchronization
Amount of participating system:
1:1
1:n
Amount of exchanged data:
Incremental
Usage of timestamps, attached to data items
Comparison of sub data structures and exchange
Full
Conflict resolution mechanisms
Questions: who wins ?
Merging
System time synchronization in distributed systems
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 8

9. Merging Data During Synchronization: The Product - Sales Example
System A
System B
System A
System B
Order 50 units
of product A
Order 25 units
of product A
Order Product A: 50
Order Product A: 25
Stock Product A: 100
Stock Product A: 100
Stock Product A: 50
Stock Product A: 75
No communiation
Apply updates
Main System
Main System
Stock Product A: 25
Stock Product A: 100
Reduce stock,
Insert order
Reduce stock,
Insert order
No conflict, minimum stock is preserved
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 9

10. Synchronization solutions
File system oriented:
Unix tool rsync:
Incremental exchange of files
Coda file system
Version control systems:
CVS
WEBDav protocol:
HTTP based file transfer protocol
Locking and Versioning included
Mobile Application oriented:
Microsoft Active Sync:
Synchronization between Desktop and Windows CE mobile devices
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 10

11. Standardization Efforts in Data Synchronization: SyncML
SyncML defines client server based protocol for synchronization
SyncML is XML based, main elements:
SyncMessage
Sync Commands:
Modification commands: add, copy, delete, put and replace
Query commands: read and search
Response commands: status and results
Execution specific commands: sequence and atomic
Synchronization models:
Two way sync
Slow sync
One way sync
Refresh sync
Server-alerted sync
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 11

12. SyncML Architecture
Source: SyncML Representation Protocol, version 1.0.1,
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 12

13. Agenda Motivation Mobile Computing Synchronization Techniques
Replication Techniques in Databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 13

14. What is Replication ?
Usage of multiple copies of server system (replicas)
Goals:
Higher availability
Better performance
Areas of usage: Distributed Systems and Databases
Replicated Server Only
Ressource
Server 1
Server n
Replicated Ressource and Server
Ressource n
Ressource 1
Server n
Server 1
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 14

15. Replication in Databases
Replication of the resource replicating the database
Elimination of single point of failure
Single server behaviour
Correctness criterion: one-copy serializability
Synchronization in databases: replication
Constant trade-off between consistency and efficiency
Update anywhere, anytime, any way replication scheme
Simple case:
Only a few replica nodes
Low transaction rates
Small amount of deadlocks and reconciliation
System scale up:
Problem complexity grows drastically at cubic rates
Inconsistent replicas and system delusion (no way to repair the inconsistencies)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 15

16. Replication in Databases – A Classification Scheme
Classification dimensions:
Where
Dedicated replica:
Single Master (primary master): primary and secondaries
Every replica:
Multi Master (update everywhere)
When
Synchronous (eager replication)
Asynchronous (lazy replication)
Other (not considered here)
Interaction activities (network traffic)
Constant or linear growth
Transaction termination:
Voting or Non-Voting techniques
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 16

17. Abstract Functional Replication Model
Distinction between different replication solutions:
Phase ordering
Phase skipping
Request
Phase:
Client sends a
request to
one or
all replicas
Server
Coordination
Phase:
Replicas
coordinate
their work
with each other
(ordering of
Execution
steps)
Execution
Phase:
Replicas
execute
operation
Agreement
Coordination
Phase:
Replicas ensure
that they all
have done
the same
thing (2pc)
Client
Response
Phase:
Client receives
a response
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 17

18. Synchronous Single Master Replication I
Characteristics
Keeps all replicas synchronous in consistent state
No update anomalies
Reduces performance
Increase response times
Possible solution
Trigger on selected tables
Updates remotely all replicas
Fails if one replica fails
Usage of 2pc
Implication:
No control over when replication should take place
Only databases with 2pc support
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 18

19. Synchronous Single Master Replication II
(primary)
(secondary)
(secondary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 19

20. Asynchronous Single Master Replication I
Characteristics
Response after primary processed request:
Primary takes care about transaction ordering
Drawback: stale data items versions
Performance is main goal
Possible solutions
Database system log based propagation:
Committed updates only
Restriction: Wait for current transactions
Further improvement: only send data fields
Log table:
Trigger records update and insert actions
Periodically recreate log table
Send log table to replicas (preserving oder)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 20

21. Asynchronous Single Master Replication II
(primary)
(secondary)
(secondary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 21

22. Handling Failures Two types of failure: Replica fails:
Secondary
System not affected
Secondary catch up
Primary
Re-election of primary
Most up-to-date replica: common system state
Possible data loss
Network partition
Problem: Which partition is allowed to process further requests ?
Majority consensus algorithms
Quorum consensus algorithms
Danger: Dramatic system performance loss
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 22

23. Synchronous Multi Master Replication I
Similar to single master synchronous replication
Updates allowed at every replica
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 23

24. Synchronous Multi Master Replication II
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 24

25. Asynchronous Multi Master Replication I
Characteristics
Multi version concurrency control mechanisms
Conflicts
Reconciliation
Automatic
Manually
Possible solution
Unique timestamps
Concatenation of local replica system clock to replica identifier
Thomas Write Rule:
Update applied if timestamp numerical greater than local timestamp
Also known as optimistic replication
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 25

26. Asynchronous Multi Master Replication II
(primary)
(primary)
(primary)
Source: Wiesmann, Pedone, Schiper, Kemme, Alonso: Understanding Replication in Databases and Distributed Systems,
Proceedings of 20th International Conference on Distributed Computing Systems (ICDCS'2000)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 26

27. Replication Requirements in Mobile Context
Disconnected operations require
Asynchronous Multi Master Replication
More frequent reconciliation because of
Lazy replication
Message delays
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 27

28. Replication Techniques - Summary
Replication classification scheme
Update propagation
Eager / synchronous
primary server
Lazy / asynchronous
Update everywhere
Lazy / asynchromnous
Update location
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 28

29. Agenda Motivation Mobile Computing Synchronization techniques
Replication techniques in databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 29

30. Oracle Lite 10g – A Mobile Database Solution
Supports disconnected mobile database applications
Local database (snapshot) on mobile device
Content: subset of enterprise database
Synchronization with enterprise database
Central component: Mobile Server
Application Model
Publication
Deployed on Mobile Server
Distribution of application code
Distribution of SQL defined snapshots (controllable by variable)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 30

31. Oracle Lite 10g Architecture
Source: Oracle® Database Lite Developer's Guide 10g (10.0.0) Part No. B
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 31

32. Oracle Lite 10g Synchronization Process I
Snapshot
Keeps track of local modifications
Three types of publication items (application depended)
Complete refresh
All data items
Fast refresh
Incremental (updates and inserts)
Queue based refresh
No refresh (e.g. data collection: newly created data items)
Asynchronous synchronization
Central component Mobile Server
Via „In queues“ and „Out queues“
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 32

33. Oracle Lite 10g Synchronization Process II
Source: Oracle® Database Lite Developer's Guide 10g (10.0.0) Part No. B
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 33

34. Oracle Lite 10g Evaluation
Well suited for „build from scratch“
Enterprise database directly accessible from Mobile Server
„Mobile only“ or „Designed for mobility“ applications
Existing data access layers not supported (EJB, CORBA, other)
Duplication of mission-critical data access logic
Possible code portation efforts
Maintenance efforts increase
Benefits reduced or outweighted
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 34

35. Agenda Motivation Mobile Computing Synchronization techniques
Replication techniques in databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 35

36. Peer to Peer Mobile Computing
Shortcommings of client server model
No true mobility
No communication between mobile devices
Requirements
Any to any communication
All devices equal
Large replication factors
Careful data replication
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 36

37. The Ward Model I Ward 2 Ward 3 Ward member Ward 1 Ward master
Overlapped ward member
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 37

38. The Ward Model II Dynamically reconfigurable synchronization topology
All units equal mobile peers:
Intra ward mobility
Inter ward mobility
Ward changing (heavyweight)
Ward overlapping (lightweight)
Ward master:
Re-electable
Knows ward members
No storage of ward member data
Manages data import / export for ward
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 38

39. The Ward Model III Second, top-level ward: all ward master
Ward set: data of all ward members within one ward
Changes over time
Optimistically replicated within ward
Selective replication: intra- and inter-ward
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 39

40. Two Tier Replication Characteristics Comparable with ward model
Static backbone of stationary replicated database servers
Mobile replicas connect eventually at all stationary replicas
Benefit
Compromise between efficiency and true mobility
Possible improvement:
Combination with ward model (selective replication)
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 40

41. Agenda Motivation Mobile Computing Synchronization techniques
Replication techniques in databases
Oracle Lite 10g – A Mobile Database Solution
Peer to Peer Mobile Computing
Future Applications and Developments
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 41

42. Future Applications and Developments
Car to car communication (
Mobile network of communicating cars
Security services
Development of replication schemes, based on ward model ?
Moving wards
Multi level ward hierachy
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 42

43. Thanks for your attention !
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 43

44. Discussion
Try to describe other real world usage scenarios of mobile applications, where synchronization and replication techniques need to be used
Which technologies could be helpful for application level data replication and integration. Think of technologies in the field of Enterprise Application Integration (EAI). Which advantages and disadvantages can be identified with respect to requirements in mobile applications
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 44

45. Appendix I: SyncML Example
Source: SyncML Representation Protocol, version 1.0.1,
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 45

46. Appendix II: Majority Consensus Algorithm I
Secondary 1
Secondary 1
Secondary 2
Network
Partition
Secondary 2
Primary
Primary
Secondary 3
Secondary 4
Secondary 4
Secondary 3
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 46

47. Appendix II: Majority Consensus Algorithm II
Secondary 2
Secondary 1
2,5 < 2 3
Inactive
Partition
Working
partition
Old Primary
New Primary
Secondary 4
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 47

48. Appendix III: Quorum Consensus Algorithm I
Weight: 1
Secondary 1
Weight: 1
Weight: 2
Secondary 1
Primary
Primary
Weight: 2
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 48

49. Appendix III: Quorum Consensus Algorithm II
>
Partition weight = 2
Partition weight = 1
Working
partition
Weight: 2
Inactive
partition
Weight: 1
Primary
Secondary 1
JASS 05 Synchronization and Replication in the Context of Mobile Applications
© Alexander Stage 49