Using Geodatabase Replication To Manage Distributed Data David Best Redwood National Park Craig Dalby Pacific West Regional Office 2008 GIS & Data Management Conference National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

Contents The Problem Solutions and Consequences Geodatabase Replication Pilot Project Installing and Configuring SDE Services Managing Replication Results and Recommendations Next Steps 2008 GIS & Data Management Conference National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

The “Problem ” A great challenge is how parks will obtain and edit national GIS datasets. Options include moving data (and applications) to national servers, or… Keeping data local, and pushing changes to national datasets 2008 GIS & Data Management Conference National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

Centralized Data Field (remote) users give up local applications and local data in favor of Remote Desktop/Citrix, or… Repeatedly download and upload data between local servers and national servers Few organizations have the network bandwidth to accommodate local applications directly editing remote data National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Typical NPS Environment For most parks there is a dramatic performance improvement when working with local data. Local data makes it easier to customize user interfaces, more efficiently access other data stored on park servers, and integrate data of park-only interest. National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Geodatabase Replication It is inconvenient to move data back and forth from the edit (local server) and integrated (national server) environments. ArcGIS 9.2 introduced new technology for this problem: Distributed Geodatabase. Allows versioned SDE geodatabases to be deployed at multiple locations. Changes are synchronized with replica versions using either one-way or two- way replication. National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Benefits A park can house and maintain data for a specific geographic area, gaining the performance of “local” data. Seamlessly, these data contribute to national datasets. Changes made at either the parent (EGIS) or park (child) can be replicated to the other. This allows parks and programs to retain stewardship responsibilities, while ensuring an authoritative national versions are maintained. National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Need to Evaluate Geodatabase Replication is new – want to test stability and reliability NPS computing environment is diverse – want to test suitability in small parks with no IT support and large parks with dedicated GIS specialists and servers National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Types of Replication Checkout- Checkin One Way Two Way Parent Child Parent Child Multiple Times One Time National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Replication Options Conserve bandwidth by –Re-use schema Where target already has data and only needs updating –Schema only Mobile users who input only new data –Register Existing Data Bypass data copy step and use existing data –Replicate related data Include or not include relationship classes National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

About Synchronization Synchronization –Synchronization is when one replica sends data changes and the relative replica receives changes. –Changes include inserts, updates and deletes made to the replica version. –Must be connected as the same database user that was used to create the replica Filters and Relationship Classes –Filters and classes that were applied during creation of the replica are also applied during synchronization –Data transfer based on exchanging replication messages –Detect and automatically recover from errors by unreliable messaging. National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

PWRO Project Funds Participating Parks: RedwoodLassen Volcanic Lava BedsOregon Caves Sequoia-KingsPacific West RO Had to overcome reluctance. Difficult for a park to see benefit. Common question: what am I going to have to do? National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

Objectives Collect and enter building footprints Develop methods and procedures to install and configure two-way replication Implement two-way replication and train park staff Evaluate performance and make recommendations for EGIS use. Can an out-of-the box standardized ArcSDE deployment be seamlessly and consistently deployed to park offices? National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference General Steps 1.Install SQL Server 2.Install SDE 3.Perform SDE post-install 4.Configure using Access Database 5.Create Replica on child 6.Schedule Synchronization

Configure Two-way Replication MS Access database with embedded VB code is used to automate: –Record information about location, IP addresses, server name, database locations –Create new Active Directory groups for administration, editing, and viewing park- based EGIS data –Create folders/shares for administrative files –Create SDE connection files –Enable SQL backup –Implement SQL database roles and AD security National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference 1

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference 2

OTHER PARKS Lassen Volcanic PWRO Lava Beds Redwood EGIS Database Produces Consistent Share Name with AD Security National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 3

Create database EGIS (SDE created by SDE install) Create logins for Administrators Editors Viewers

Set permissions for SDE and EGIS databases Create Database Roles: Editors Viewers

Connects to EGIS database as the LABE SDE user on the park’s SDE server Connects to parks version on the EGIS database as the LABE SDE user on the EGIS server EGIS EGIS Database Park Version as park SDE use PARK EGIS Database Default Version as park SDE user National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 4

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference SDE Connection Files Can Be Read By ArcCatalog

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region Right-click to view “Connection Properties”

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region Connection Properties

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 5

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 6

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 7

Lassen Volcanic PWRO Lava Beds OTHER PARKS Automated Backup of SQL Databases Are Consistent National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 8

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference Python Script for Synchronization

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference Create Replica on Child In most cases an EGIS responsibility Data is mainly “pulled” instead of “pushed”. No need to disclose EGIS security to parks. ArcMap provides spatial filter to limit replica to park’s geographic area (not available in ArcCatalog) Filters are great way to eliminate versions and posting

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region Activate Distributed Geodatabase Toolbar

Load data that will participate in replica into map

Specify Type of Replica

Specify Connection and Replica Name REPLICA OWNER PARENT CHILD

Use to limit geographic area where synchronization occurs. Useful for avoiding SDE versions Use for trusted editors and offices with oversight responsibility Specify Spatial Extent

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference Managing Replication Synchronization can be manual or automated. Synchronization begins for each park as a separate task within “Schedule Tasks.” The task is a batch (.bat) file that calls the python script created earlier. FROM “EGIS” TO:

National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region 2008 GIS & Data Management Conference Monitoring Replication Right click SDE database  Distributed Geodatabase  Manage Replicas

Viewing the Replication Log Replication Manager Replication Log

Results and Recommendations Complete install about 3 hrs/server Get Administrator access Other Flavors of SQL/SDE probably not worth coast savings SQL backup very easy and reliable SDE Service needs monitoring Two-way replication avoids permission management with versions Replica design should be updatable Infrastructure mostly adequate 2008 GIS & Data Management Conference National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

Results and Recommendations Have not trained park users, but do not expect difficulties. Unforeseen benefit may be rapid- deployment of region or network- based integrated datasets. Datastore should consider one-way “data subscription” service for national datasets not edited by parks. Annotation is an easy and very useful national test dataset GIS & Data Management Conference National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region

Next Steps Complete Klamath Network Parks. Encourage frequent park editing. Review by EGIS Team. Add new theme. Move parent to RIMD. Work on VPN solution. ELA for SQL Server. New participants? 2008 GIS & Data Management Conference National Park Service U.S. Department of the Interior Redwood National Park Pacific West Region