1. Replication Monitoring University of Maryland Institute for Advanced Computer Studies

2. TPAP Replication Problem  5.8 million files  Copy to three sites  Different collections are stored on different accounts, different locations

3. SRB Replication Monitor  Portal to view and manage multiple collections  Tracks all events that occur on a collection New files found, copies made, missing files  Generate a report of missing, non- replicated files.

4. Lessons Learned  Expect failure in large collections DB crashed Network maintenance Disk failure Heisenbugs  Average failure rate was 1-2% of all copy attempts  All parts of replication must be independent  Restart replication on error is not an option Restarts took over a week Allow replication to opportunistically continue scan

5. Demo

6. Main Screen

7. Logging

8. Browse Collection

9. Report Missing Items

10. Configuring Replicas