1. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 1 Digital Networking TOI David Smith (davesm@cisco.com)

2. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 2 Digital Networking  Unifies the Directory  No Redundancy  Connected Nodes

3. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 3 Technology  Object changes are sent to remote nodes via SMTP  The body of the messages is XML  Voice names are replicated as MIME attachments to the SMTP message.  The “Connection Digital Networking Replication Agent” is the service that handles sending and receiving changes. This service is deactivated initially and must be activated after joining nodes together.  Messages are digitally signed by the sender to ensure authenticity.

4. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 4 Terminology  USN Unique sequence number Identifies each change to a replicated object on the local node. Each replicated change includes the USN for the object to allow the remote node to acknowledge receipt of the change. Replication progress with a remote node is tracked by three USN values: Last USN Received Last USN Sent Last USN Acknowledged

5. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 5 Terminology (cont’d)  Replication Set When paired with a USN they uniquely identify a specific change. When the replication set changes due to an upgrade, restore, or rollback, the object changes will be reordered to make sure they accurately reflect the objects on the node. When remote nodes see a replication set change, they will ignore changes from earlier replication sets.

6. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 6 Replicated Objects  VMS Locations  Partitions  Search Spaces  VPIM Locations  Users  Contacts  Distribution Lists & Members

7. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 7 Message Flow SMTP Server Replicator SMTP Server Replicator network/ drop smtp/default/ networkpickup Base path: /var/opt/cisco/connection

8. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 8 Lifecycle  Joining Nodes  Push Synchronization  Pull Synchronization  Normal Replication  Removing a Node

9. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 9 Joining Nodes  Nodes can be joined in two different ways: Automatic By specifying the remote node address, and administrator account credentials, we register the local node remotely and the remote node locally. Manual By downloading the local configuration and then uploading this configuration to the remote node, we accomplish the same registration as through the automatic method. You must upload the remote configuration file to the local node for both nodes in the network. This method is useful when HTTPS traffic is blocked between nodes, making the automatic method inoperable.

10. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 10 Joining Nodes (cont’d)  When adding a third, fourth or fifth node to a network it is only necessary to join the node to one other node in the existing network.  Nodes that did not participate in the join may be delayed (default: 5 minutes) in synchronizing with the newly added node.

11. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 11 Push Synchronization  Required when things get out of whack between nodes. Out of whack may be indicated by: USNs that are not advancing Users’ inability to address to remote objects  Pushes all replicated objects from the local node to the targeted remote node.  After upgrade, restore or rollback, we will automatically initiate a push to all remote nodes.  The receiving side can stop a synchronization initiated from the sending side.

12. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 12 Push Synchronization (cont’d)  Object order During synchronization, we will send these objects first in order to satisfy dependencies for later objects: VMS Locations Partitions Search Spaces VPIM Locations After sending these, we will send the rest of the objects according to their USN order: Users Contacts Distribution Lists & Members

13. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 13 Push Synchronization State Machine Numbers in parentheses correspond to values in tbl_LocationVMS.PushState

14. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 14 Pull Synchronization  Identical to push synchronization except it is initiated from the receiving side rather than the sending side.  Before receiving the data the receiving side will catalog all the objects it knows belong to the sending side. When synchronization is finished, any objects in the catalog not received during synchronization will be deleted.  After upgrade, restore or rollback, we will automatically initiate a pull from all remote nodes.  The sending side can stop a synchronization initiated on the receiving side.

15. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 15 Pull Synchronization State Machine Numbers in parentheses correspond to values in tbl_LocationVMS.PullState

16. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 16 Normal Replication  After the network nodes are synchronized, we enter the normal replication state.  Changes made to replicated objects will be sent to all remote nodes periodically according to System.Networking.ReplicationInterval (configuration setting, default 15 seconds).  Each change creates a new USN for the node.  While synchronizing with a remote node, we may send new changes along with older synchronizing changes.

17. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 17 Removing a Node  Necessary when a node fails catastrophically. Avoids unnecessary replication.  Necessary to change the host name of a node. First remove the node from the network Change the host name. Join the node to the network again.  Nodes that were not directly involved in the removal may require additional time to process the node removal. For example, network A, B, C; on A, remove node C; it may take additional time for B to remove C.

18. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 18 Troubleshooting  USN Mismatch When everything is synchronized and up-to-date, the last USN sent and last USN acknowledged on the sending node should equal the last USN received on the receiving node. During replication, it is normal for the last USN acknowledged to lag behind the last USN sent. During a push synchronization, the last USN sent may show a very large value while the last USN acknowledge shows a much smaller value. This is normal. Monitor the last USN acknowledged to make sure it continues increasing toward the last USN sent.

19. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 19 Troubleshooting  Stalled Replication If the last USN acknowledged on the sending side, or the last USN received on the receiving side stops incrementing toward last USN sent, replication may be stalled. Wait for System.Networking.DependencyTimeout (configuration value, default 5 minutes) time to see if the values increment. Check the system log to see if anything has been reported as a problem in replication. Check the replicator log to diagnose the issue.

20. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 20 Troubleshooting  Stalled Replication The push or pull status may indicate replication is In Progress, but the last USN acknowledged or last USN received may not be changing. Try stopping the push or pull operation.

21. © 2008 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 21 Troubleshooting  Push/Pull Status Mismatch The push status on the sending node should match the pull status on the receiving node. If they are different and the In Progress status doesn’t change to Idle or the Idle doesn’t change to In Progress after System.Networking.DependencyTimeout (default: 5 minutes), you may be able to correct it by doing this: On the node showing Idle start the opposite operation showing In Progress on the remote node. Example: Node 1 shows a push in progress and Node 2 shows pull is idle. On Node 2, start a pull from Node 1. After the operation shows In Progress, wait a minute and then stop the operation.