1. Distributed Systems CS 15-440
Naming (Last Part) & Synchronization (Introduction)
Lecture 9, Sep 22, 2014
Mohammad Hammoud

2. Today… Last Session Naming (Cont’d) Today’s Session Synchronization
Coordinated Universal Time (UTC)
Tracking Time on a Computer
Clock Synchronization: Cristian’s Algorithm
Announcements
Quiz 1 is “this week” on Thursday, Sep 25, during the recitation time
Project 1 is due next week on Wed, Oct 1st, 2014 by midnight (it is worth 15% of the overall grade!)

3. Key Resolution in Chord: An Example1 04 2 3 09 4 5 18
succ(p + 2(i-1)) i 1 01 2 3 4 04 5 14 1 09 2 3 4 14 5 20 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 12 1 11 2 3 14 4 18 5 28 1 28 2 3 4 01 5 09 1 14 2 3 18 4 20 5 28 1 21 2 28 3 4 5 04 1 18 2 3 4 28 5 01 1 20 2 3 28 4 5 04

4. Chord – Join and Leave Protocol
In large Distributed Systems, nodes dynamically join and leave (voluntarily or due to failure)
If a node p wants to join:
Node p contacts arbitrary node, looks up for succ(p+1), and inserts itself into the ring
If node p wants to leave
Node p contacts pred(p), and updates it 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Node 4 is succ(2+1)
Who is succ(2+1) ? 02

5. Chord – Finger Table Update Protocol
For any node q, FTq[1] should be up-to-date
It refers to the next node in the ring
Protocol:
Periodically, request succ(q+1) to return pred(succ(q+1))
If q = pred(succ(q+1)), then information is up-to-date
Otherwise, a new node p has been added to the ring such that q < p < succ(q+1)
FTq[1] = p
Request p to update pred(p) = q
Similarly, node p updates each entry i by finding succ(p + 2(i-1))

6. Classes of Naming
Flat naming
Structured naming
Attribute-based naming

7. Structured Naming
Structured Names are composed of simple human-readable names
Names are arranged in a specific structure
Examples
File-systems utilize structured names to identify files
/home/userid/work/dist-systems/naming.txt
Websites can be accessed through structured names

8. Name Spaces Structured Names are organized into name spaces
Name-spaces is a directed graph consisting of:
Leaf nodes
Each leaf node represents an entity
Leaf node generally stores the address of an entity (e.g., in DNS), or the state of an entity (e.g., in file system)
Directory nodes
Directory node refers to other leaf or directory nodes
Each outgoing edge is represented by (edge label, node identifier)
Each node can store any type of data
e.g., type of the entity, address of the entity

9. Looking up for the entity with name “/home/steen/mbox”
Example Name Space
Looking up for the entity with name “/home/steen/mbox”
Data stored in n1 n0
keys
home
n2: “elke”
n3: “max”
n4: “steen” n1 n5
“/keys”
elke
steen
max n4 n2 n3
Leaf node
twmrc
mbox
Directory node

10. Name Resolution
The process of looking up a name is called Name Resolution
Closure mechanism
Name resolution cannot be accomplished without an initial directory node
Closure mechanism selects the implicit context from which to start name resolution
Examples
start at the DNS Server
/home/steen/mbox: start at the root of the file-system

11. Name Linking
Name space can be effectively used to link two different entities
Two types of links can exist between the nodes
Hard Links
Symbolic Links

12. “/home/steen/keys” is a hard link to “/keys”
1. Hard Links
There is a directed link from the hard link to the actual node
Name Resolution
Similar to the general name resolution
Constraint:
There should be no cycles in the graph
“/home/steen/keys” is a hard link to “/keys” n0
home
keys n1 n5
“/keys”
elke
steen
max
keys n4 n2 n3
twmrc
mbox

13. “/home/steen/keys” is a symbolic link to “/keys”
2. Symbolic Links
Symbolic link stores the name of the original node as data
Name Resolution for a symbolic link SL
First resolve SL’s name
Read the content of SL
Name resolution continues
with content of SL
Constraint:
No cyclic references should be present
“/home/steen/keys” is a symbolic link to “/keys” n0
home
keys n1 n5
“/keys”
elke
steen
max n4 n2 n3
keys
twmrc
mbox n6
Data stored in n6
“/keys”

14. Mounting of Name Spaces
Two or more name spaces can be merged transparently by a technique known as mounting
In mounting, a directory node in one name space will store the identifier of the directory node of another name space
Network File System (NFS) is an example where different name spaces are mounted
NFS enables transparent access to remote files

15. Example of Mounting Name Spaces in NFS
Machine A
Machine B
Name Server for
foreign name space
Name Space 1
Name Space 2
remote
home vu
steen
mbox
“nfs://flits.cs.vu.nl/home/steen” OS OS
Name resolution for “/remote/vu/home/steen/mbox” in a distributed file system

16. Distributed Name Spaces
In large Distributed Systems, it is essential to distribute name spaces over multiple name servers
Distribute nodes of the naming graph
Distribute name space management
Distribute name resolution mechanisms

17. Layers in Distributed Name Spaces
Distributed Name Spaces can be divided into three layers
Global
Layer
Consists of high-level directory nodes
Directory nodes are jointly managed by different administrations
Administrational Layer
Contains mid-level directory nodes
Directory nodes grouped together in such a way that each group is managed by an administration
Managerial Layer
Contains low-level directory nodes within a single administration
The main issue is to efficiently map directory nodes to local name servers

18. Distributed Name Spaces – An Example

19. Comparison of Name Servers at Different Layers
Global
Administrational
Managerial
Geographical scale of the network
Total number of nodes
Number of replicas
Update propagation
Is client side caching applied?
Responsiveness to lookups
Worldwide
Organization
Department
Few
Many
Vast numbers
Many
None or few
None
Lazy
Immediate
Immediate
Yes
Yes
Sometimes
Seconds
Milliseconds
Immediate

20. Distributed Name Resolution
Distributed Name Resolution is responsible for mapping names to addresses in a system where:
Name servers are distributed among participating nodes
Each name server has a local name resolver
We will study two distributed name resolution algorithms:
Iterative Name Resolution
Recursive Name Resolution

21. 1. Iterative Name Resolution
Client hands over the complete name to root name server
Root name server resolves the name as far as it can, and returns the result to the client
The root name server returns the address of the next-level name server (say, NLNS) if address is not completely resolved
Client passes the unresolved part of the name to the NLNS
NLNS resolves the name as far as it can, and returns the result to the client (and probably its next-level name server)
The process continues till the full name is resolved

22. 1. Iterative Name Resolution – An Example
<a,b,c> = structured name in a sequence
#<a> = address of node with name “a”
Resolving the name “ftp.cs.vu.nl”

23. 2. Recursive Name Resolution
Approach
Client provides the name to the root name server
The root name server passes the result to the next name server it finds
The process continues till the name is fully resolved
Drawback:
Large overhead at name servers (especially, at the high-level name servers)

24. 2. Recursive Name Resolution – An Example
<a,b,c> = structured name in a sequence
#<a> = address of node with name “a”
Resolving the name “ftp.cs.vu.nl”

25. Classes of Naming
Flat naming
Structured naming
Attribute-based naming

26. Attribute-based Naming
In many cases, it is much more convenient to name, and look up entities by means of their attributes
Similar to traditional directory services (e.g., yellow pages)
However, the lookup operations can be extremely expensive
They require to match requested attribute values, against actual attribute values, which requires inspecting all entities
Solution: Implement basic directory service as database, and combine with traditional structured naming system
We will study Light-weight Directory Access Protocol (LDAP); an example system that uses attribute-based naming

27. Light-weight Directory Access Protocol (LDAP)
LDAP Directory Service consists of a number of records called “directory entries”
Each record is made of (attribute, value) pairs
LDAP standard specifies five attributes for each record
Directory Information Base (DIB) is a collection of all directory entries
Each record in a DIB is unique
Each record is represented by a
distinguished name
e.g., /C=NL/O=Vrije Universiteit/OU=Comp. Sc.

28. Directory Information Tree in LDAP
All the records in the DIB can be organized into a hierarchical tree called Directory Information Tree (DIT)
LDAP provides advanced search mechanisms based on attributes by traversing the DIT
Example syntax for searching all Main_Servers in Vrije Universiteit:
search("&(C = NL) (O = Vrije Universiteit) (OU = *) (CN = Main server)")

29. Summary
Naming and name resolutions enable accessing entities in a Distributed System
Three types of naming
Flat Naming
Home-based approaches, Distributed Hash Table
Structured Naming
Organizes names into Name Spaces
Distributed Name Spaces
Attribute-based Naming
Entities are looked up using their attributes

30. Synchronization Until now, we have looked at:
how entities communicate with each other
how entities are named and identified
In addition to the above requirements, entities in DSs often have to cooperate and synchronize to solve a given problem correctly
e.g., In a distributed file system, processes have to synchronize and cooperate such that two processes are not allowed to write to the same part of a file

31. Need for Synchronization – Example 1
Vehicle tracking in a City Surveillance System using a Distributed Sensor Network of Cameras
Objective: To keep track of suspicious vehicles
Camera Sensor Nodes are deployed over the city
Each Camera Sensor that detects a vehicle reports the time to a central server
Server tracks the movement of the suspicious vehicle
1:08 PM:
Car spotted
1:12 PM:
Car spotted
1:05 PM:
Car spotted
1:16 PM:
Car spotted
If the sensor nodes do not have a consistent version of the time, the vehicle cannot be reliably tracked
1:15 PM:
Car spotted
1:25 PM:
Car spotted
1:00 PM:
Car spotted
3:00 PM:
Car spotted
1:18 PM:
Car spotted
1:17 PM:
Car spotted

32. Need for Synchronization – Example 2
Writing a file in a Distributed File System
Client A
Server
Client C
Write data1 to file abc.txt at offset 0
Distributed
File
abc.txt
Write data3 to file abc.txt at offset 1
Client B
Write data2 to file abc.txt at offset 1
If the distributed clients do not synchronize their write operations to the distributed file, then the data in the file can be corrupted

33. A Broad Taxonomy of Synchronization
Reason for synchronization and cooperation
Examples
Requirement for entities
Topics we will study
Entities have to agree on ordering of events
Entities have to share common resources
e.g., Vehicle tracking in a Camera Sensor Network, Financial transactions in Distributed eCommerce Systems
e.g., Reading and writing in a Distributed File System
Entities should have a common understanding of time across different computers
Entities should coordinate and agree on when and how to access resources
Time Synchronization
Mutual Exclusion

34. Overview Time Synchronization Mutual Exclusion Election Algorithms
Today’s lecture
Time Synchronization
Physical Clock Synchronization (or, simply, Clock Synchronization)
Here, actual time on computers are synchronized
Logical Clock Synchronization
Computers are synchronized based on relative ordering of events
Mutual Exclusion
How to coordinate between processes that access the same resource?
Election Algorithms
Here, a group of entities elect one entity as the coordinator for solving a problem
Next two lectures

35. Overview Time Synchronization Mutual Exclusion Election Algorithms
Clock Synchronization
Logical Clock Synchronization
Mutual Exclusion
Election Algorithms

36. Clock Synchronization
Clock Synchronization is a mechanism to synchronize the time of all the computers in a DS
We will study
Coordinated Universal Time
Tracking Time on a Computer
Clock Synchronization Algorithms
Cristian’s Algorithm
Berkeley Algorithm
Network Time Protocol

37. Clock Synchronization
Coordinated Universal Time
Tracking Time on a Computer
Clock Synchronization Algorithms
Cristian’s Algorithm
Network Time Protocol
Berkeley Algorithm

38. Coordinated Universal Time (UTC)
All the computers are generally synchronized to a standard time called Coordinated Universal Time (UTC)
UTC is the primary time standard by which the world regulates clocks and time
UTC is broadcasted via the satellites
UTC broadcasting service provides an accuracy of 0.5 msec
Computer servers and online services with UTC receivers can be synchronized by satellite broadcasts
Many popular synchronization protocols in distributed systems use UTC as a reference time to synchronize clocks of computers

39. Clock Synchronization
Coordinated Universal Time
Tracking Time on a Computer
Clock Synchronization Algorithms
Cristian’s Algorithm
Berkeley Algorithm
Network Time Protocol

40. Tracking Time on a Computer
How does a computer keep track of its time?
Each computer has a hardware timer
The timer causes an interrupt ‘H’ times a second
The interrupt handler adds 1 to its Software Clock (C)
Issues with clocks on a computer
In practice, the hardware timer is imprecise
It does not interrupt ‘H’ times a second due to material imperfections of the hardware and temperature variations
The computer counts the time slower or faster than actual time
Loosely speaking, Clock Skew is the skew between:
the computer clock and the actual time (e.g., UTC)

41. Clock Skew
When the UTC time is t, let the clock on the computer have a time C(t)
Three types of clocks are possible
Perfect clock:
The timer ticks ‘H’ interrupts a second
dC/dt = 1
Fast clock:
The timer ticks more than ‘H’ interrupts a second
dC/dt > 1
Slow clock:
The timer ticks less than ‘H’ interrupts a second
dC/dt < 1
dC/dt > 1
dC/dt = 1 15
Perfect clock
Fast Clock 10
Clock time, Cp(t)
Slow Clock 7
dC/dt < 1 10
UTC, t

42. Clock Skew (cont’d)
Frequency of the clock is defined as the ratio of the number of seconds counted by the software clock for every UTC second
Frequency = dC/dt
Skew of the clock is defined as the extent to which the frequency differs from that of a perfect clock
Skew = dC/dt – 1
Hence,
dC/dt > 1
dC/dt = 1
Perfect clock
Fast Clock
Clock time, Cp(t)
Slow Clock
dC/dt < 1
UTC, t

43. Maximum Drift Rate of a Clock
The manufacturer of the timer specifies the upper and the lower bound that the clock skew may fluctuate. This value is known as maximum drift rate (ρ)
How far can two clocks drift apart?
If two clocks are drifting from UTC in the opposite direction, at a time Δt after they were synchronized, they may be as much as 2ρΔt seconds apart
Guaranteeing maximum drift between computers in a DS
If maximum drift permissible in a DS is δ seconds, then clocks of every computer must be resynchronized at least every δ/2ρ seconds
1 – ρ <= dC/dt <= 1 + ρ

44. Clock Synchronization
Coordinated Universal Time
Tracking Time on a Computer
Clock Synchronization Algorithms
Cristian’s Algorithm
Berkeley Algorithm
Network Time Protocol

45. Cristian’s Algorithm
Flaviu Cristian (in 1989) provided an algorithm to synchronize networked computers with a time server
The basic idea:
Identify a network time server that has an accurate source for time (e.g., the time server has a UTC receiver)
All the clients contact the network time server for synchronization
However, the network delays incurred while the client contacts the time server will have outdated the reported time
The algorithm estimates the network delays and compensates for it

46. Cristian’s Algorithm – Approach
Client Cli sends a request to Time Server Ser, time stamped its local clock time T1
Ser will record the time of receipt T2 according to its local clock
dTreq is network delay for request transmission
Time Server Ser T2 T3 T1
T1, T2, T3 T1 T4
Client Cli
dTreq
dTres
Ser replies to Cli at its local time T3, piggybacking T1 and T2
Cli receives the reply at its local time T4
dTres is the network delay for response transmission
Now Cli has the information T1, T2, T3 and T4
Assuming that the transmission delay from CliSer and SerCli are the same
T2-T1 ≈ T4–T3

47. Christian’s Algorithm – Synchronizing Client Time
Client C estimates its offset θ relative to Time Server S
θ = T3 + dTres – T4
= T3 + ((T2-T1)+(T4-T3))/2 – T4
= ((T2-T1)+(T3-T4))/2
If θ > 0 or θ < 0, then the client time should be incremented or decremented by θ seconds T2 T3 S C T1 T4
dTreq
dTres
Gradual Time Synchronization at the client
Instead of changing the time drastically by θ seconds, typically the time is gradually synchronized
The software clock is updated at a lesser/greater rate whenever timer interrupts
Note: Setting clock backward (say, if θ < 0)is not allowed in a DS since decrementing a clock at any computer has adverse effects on several applications (e.g., make program)

48. Cristian’s Algorithm – Discussion
1. Assumption about packet transmission delays
Cristian’s algorithm assumes that the round-trip times for messages exchanged over the network is reasonably short
The algorithm assumes that the delay for the request and response are equal
Will the trend of increasing Internet traffic decrease the accuracy of the algorithm?
Can the algorithm handle delay asymmetry that is prevalent in the Internet?
Can the clients be mobile entities with intermittent connectivity?
Cristian’s algorithm is intended for synchronizing computers within intranets
2. A probabilistic approach for calculating delays
There is no tight bound on the maximum drift between clocks of computers
3. Time server failure or faulty server clock
Faulty clock on the time server leads to inaccurate clocks in the entire DS
Failure of the time server will render synchronization impossible

49. Summary Physical clocks on computers are not accurate
Clock synchronization algorithms provide mechanisms to synchronize clocks on networked computers in a DS
Computers on a local network use various algorithms for synchronization
Some algorithms (e.g, Cristian’s algorithm) synchronize time by contacting centralized time servers
Some algorithms (e.g., Berkeley algorithm) synchronize in a distributed manner by exchanging the time information on various computers (we’ll start with next time)