1. MarketNet Directory Services (MDS) Weibin Zhao Henning Schulzrinne Department of Computer Science Columbia University

2. MDS2 l Replication: fault tolerance, availability and performance Directory Services Client Server 1Server 2 QueryRegistration Advertisement Control Message Registration Propagation ReplyAcknowledge Server 3 Server Group

3. MDS3 MDS: MarketNet Directory Services * Approach l built on top of an existing directory service system í SLP: Service Location Protocol í lower cost, can be reused l enhance SLP reliability and scalability í adjust to service dynamic changes and losses í scale to large systems * Critical issues l data organization and distribution l handling failures: server crashes and network partitions l server group management

4. MDS4 MDS Design Overview * Peering architecture í multiple server groups * Asynchronous replication í high availability: anti-entropy (eventual consistency) * Direct forwarding í propagate new updates rapidly * Automated server group management í server advertisement & request * Dynamic states * Version resolution * Handling deleted states

5. MDS5 Peering Architecture * Peers l share serving groups l have a persistent peering connection í a reliable FIFO communication channel: TCP * Topology l an arbitrary connected graph, multiple groups l best for a full mesh Server 1 (x,y)Server 2 (y,z) Server 3 (x,z) z y x

6. MDS6 Controlling Timestamps * acceptID l for updates propagation í acceptServer: the first replica that accepts the update í acceptTS: timestamp at acceptServer * arrivalTS l for dynamic states í purge expired states * versionTS l for version resolution í each registration only updated by one client í new version overwrites the old one

7. MDS7 Messages * Server group management í Advertisement: server domain name, serving groups í AdvertisementRequest í PeerList í KeepAlive í Bye * Updates propagation í StateReport: summary of states í BatchBegin í BatchEnd í MeshForward Extension: acceptID, versionTS (append to update messages when propagated among replicas)

8. MDS8 Peering Connections * Identify l a connection í from which an original Advertisement is received l original Advertisement í advertised domain name = sender domain name * Maintain l KeepAlive l handling failures í server crashes, network partitions

9. MDS9 Peer Management * Discover new peers l static configuration, DHCP l Advertisement: multicast, unicast l exchange PeerList with a new peer l forward a new peer Advertisement to existing peers * Remove an old peer l peering connection has been closed l KeepAlive time out l Bye received

10. MDS10 Ordering and Summary * Ordering: acceptID l total order: accepted by same server l partial order: in the system * State summary l all the latest acceptIDs í ((S1, T1), (S2, T2), … (Sn, Tn)) l carried by StateReport

11. MDS11 Updates Propagations * Anti-entropy l pull model í when StateReport is received í uni-direction: from sender to receiver l scheduling StateReport í new peer and after failures (reboot and network partitions) í send to one peer at a time * Direct forwarding l push model l propagate new updates quickly í one-hop: from acceptServer to its peers

12. MDS12 Transfer Modes * Normal mode l ordering transfer * Batch Mode l unsorted states between BatchBegin and BatchEnd l deliver summary after a whole batch has been received NormalBatch new peer receive BatchBegin receive BatchEnd Anti-entropy Direct forwarding new peerafter an anti-entropy Sending Receiving

13. MDS13 Performing Updates * Version control l use versionTS: new version overwrites the old one * Dynamic states l soft state: remove a state when it expires * Handling deleted states l deletion flag l propagation: (1) registration, (2) deregistration l no special purging process

14. MDS14 Survivability * Handling Failures í server crashes, network partitions l server side í KeepAlive time out í deliver a batch after BatchEnd l client side í automatically discover/select/switch servers í response time out Server 1Server 2 Client

15. MDS15 Conclusions * MDS l a robust distributed dynamic directory service l for general purposes and customized applications * Contributions l defined a lightweight directory replication protocol í mSLP - mesh-enhanced Service Location Protocol l standalone and integrated implementation in Java l Internet Draft and ICCCN’00 paper * Future work l advance mSLP to RFC

16. MDS16 Demo l Metro-north and Diablo (servers): peering l Ruihan (client) discovers servers l registration propagations l failures: Diablo crashes and network partitions í Ruihan switches to Metro-north í Diablo recovers Metro-northDiablo Ruihan