1. Tips & Tricks for Troubleshooting Geodatabase Issues at ArcGIS 10
July 13th, 2011
Tips & Tricks for Troubleshooting Geodatabase Issues at ArcGIS 10
Chet Dobbins
Nana Y. Dei
Welcome & introductions.

2. Topics: Upgrading the geodatabase (v10.0) Compressing with Replicas
SDEINTERCEPT
Additional Troubleshooting Resources
…how do I upgrade…
…we can’t compress…
One of the main topics that we wanted to present today was upgrading the geodatabase to version 10.
- How many here have already performed this task?
After we finish with the upgrading portion we will discuss compressing with replicas, and provide some insight into the SDEINTERCEPT file which can be used for troubleshooting performance and connection issues.
If there are no questions at this point we can get started…

3. Upgrading the Geodatabase

4. Enterprise Geodatabase Upgrade - ArcGIS 10.0
As many of you may know, or have heard, upgrading an enterprise geodatabase with ArcGIS 10 is no longer performed using the ArcSDE post installation tools. Starting with ArcGIS 10 and beyond, enterprise geodatabases will be upgraded using tools provided with ArcGIS desktop. This process will be very similar to what has been available for personal & file geodatabases.

5. Before Upgrading… Backup DBMS and configuration files
DBMS full backup & configuration files
DBTUNE, DBINIT, GIOMGR
Install an ArcGIS 10 client
Upgrade integrated into client
Backwards compatible
Requirement
Direct connect
Proper credentials
From a support perspective we often see that upgrades are performed on a productions system and when the process fails, or results in an unwanted behavior, you must restore the old database from a backup. It is not possible to downgrade a geodatabase. In order to ensure that you are protected, it is best to be sure that you have backed up the database and any configuration files that were used to set up the system.
Since the ArcSDE post install is no longer used for the upgrade, it will be necessary to have an ArcGIS Desktop 10 installed to perform this operation.
Finally, be sure that you are using a direct connection with the proper credentials for upgrading. These can all be found in the online help or in the installation specific to the database management system being used.

6. Prerequisite Checks No active user connections
Install
ArcGIS 10
Client
No active user connections
Correct user to perform upgrade
Required privileges for upgrade
DBMS XML support
Geodatabase schema integrity
Geodatabase configuration check
ST shape library
Multiple user schema geodatabases
Application
Server?
Upgrade
ArcSDE
Application
yes no
Pre-requisite
Check
Upgrade
Geodatabase
Schema
Esri recommends using the prerequisite check prior to upgrading your geodatabase to determine if your geodatabase is ready for upgrading. If any of these prerequisites is not met, the tool logs an error in the Results window and does not upgrade the geodatabase. This saves you from starting the upgrade only to have it fail partway through due to one of these prerequisites not having been met. If any checks fail, you must correct the problem and restart the upgrade process.
For enterprise geodatabases the prerequisite check will determine whether there are any other active connections to the geodatabase, as well as whether the connected user has the appropriate permissions required to perform the upgrade.
XML is used for some data storage with version 10, and this process will also check for if the database is enabled for this support.
The integrity of the geodatabase is checked and this will report warnings about feature classes and other geodatabase objects which may be orphaned or other similar issues.
Post
Upgrade
Configuration

7. Geodatabase Upgrade Options
Two options
Upgrade Geodatabase button in ArcCatalog
Upgrade Geodatabase: GP Tool API
UpgradeGDB_management(…)
Beginning with the ArcGIS 10 release, geodatabase upgrades must be performed using the Upgrade Geodatabase tool or Python script. Do not use the ArcSDE Post Installation wizard on Windows or the sdesetup command to upgrade the geodatabase. If you do, you will receive a message indicating the geodatabase already exists and the upgrade will not run.

8. Upgrade Resources ArcSDE Install Guides KB Articles Online Help
-installation-guides
KB Articles
Online Help
ArcSDE Geodatabase User Forum
Here are a few additional resources that can be used when you begin preparing to upgrade to ArcGIS 10.

9. Compressing with Replicas
Next we wanted to share some tips on getting the most out of a compress when replication has been implemented in an enterprise environment.
Replication uses hidden internal system versions to manage the synchronization process between replicas. Understanding the details of how these system versions are used by ArcGIS is not necessary to successfully implement geodatabase replication. However, these versions can have an effect when attempting to compress an ArcSDE geodatabase.

10. Types of Replicas Check out/ Check in Parent Child One-way Parent
Once only
Check out/ Check in
Parent
Child
Multiple times
One-way OR
Parent
Child
To get started it will be good to know what types of replication are available.
The three types of geodatabase replication are: check out/check in, one way, and two way.
For all types, data from an ArcSDE geodatabase must be used as the source for replica creation.
Multiple times
Two-way
Parent
Child

11. Compress with a One-way Replica
Default 1 2 3 4 1 2 3
Project 1
Replica 1 1 2 3
The first example that we are going to show is versioned editing and compress with a one-way replica.
In this example Replica1 is created with the default version as the replica version.
An editor begins editing and moves parcel 3 in the DEFAULT version.
Another editor adds parcel 4 in the Project 1 version.
And then a reconcile and post is performed between Project 1 and DEFAULT.
If compress is called at this point the two edits referenced by each version cannot move to the base table. The reason is because the edits are not referenced by the Replica1 system version.
When we synchronize Replica1 and changes are sent from this geodatabase to the relative replica geodatabase, the process will send the two edits to the relative replica. The relative replica should then send an acknowledgement message which automatically removes the existing system version, and a new one is created from Default.
A compress executed at this point will be much more effective since the edits are referenced by all versions, and so that will be applied to the base tables, and removed from the delta tables. 3 4 3 4
COMPRESS

12. Compress with a Two-way Replica
Parent
Child
Default 1 2 3
Default 1 2 3 4 3 4
Replica 2
Replica 2
The next example starts the same way, except that in this case a two-way replica is created with the default version as the replica version. The replica creation process creates a system version from default for Replica2 in the parent geodatabase as well as the child geodatabase.
An editor begins editing and moves parcel 3 in the DEFAULT version of the parent replica geodatabase.
Another editor adds parcel 4 in the DEFAULT version of the child replica geodatabase
Next Replica2 is synchronized where changes are sent from the parent replica to the child replica, and the changes are acknowledged. Since an acknowledgement was received by the parent, the initial system version is deleted and a new one created. With the old system version removed, a compress called at this point on the parent replica geodatabase would remove the parcel 3 edit from the delta tables and apply it to the base table.
In the child geodatabase the parcel 3 edit is applied to the replica version (DEFAULT), and the initial system version is left unchanged. Therefore a compress on the child will result in no changes removed from the delta tables.
Replica2 is synchronized again, and changes are now pushed from the child replica to the parent replica. Again the changes are acknowledged, and this time the system version in the child geodatabase is deleted and a new one is created from default. This allows a compress on the child geodatabase to move the edits of parcels 3 & 4 from the delta tables to the base tables.
A compress on the parent would still leave the edit to parcel 4 in the delta tables. In order to move the edit of parcel 4 to the base tables on the parent, another synchronize from parent to child needs to occur.
COMPRESS
COMPRESS

13. Best Practices Remove unnecessary replicas Send changes regularly
Reconcile & Post named versions before synchronizing
Full compress is ideal but not always possible
Look to achieve an effective compress

14. SDEINTERCEPT Tracing
Sometimes when troubleshooting certain connection and performance issues, you may need to dig deeper into the communication between the ArcGIS client application and the ArcSDE geodatabase. A SDEINTERCEPT file contains all the calls to ArcSDE along with their return values.

15. SDEINTERCEPT Built-in ArcSDE functionality
Logs ArcSDE client - server calls
Helps diagnose connection & performance issues
Enable SDEINTERCEPT by setting the following environment variables:
set SDEINTERCEPT=crwTf
set SDEINTERCEPTLOC=C:\temp\sde_intercept
Flag
Description c
Intercept the API command name r
Intercept the Channel broadcasts read-only w
Intercept the Channel broadcasts write-only t
Intercept log time (minute:second) T
Intercept log time (hour:minute:second) f
Intercept flush immediate
The SDEINTERCEPT trace can be enabled on either the client or the server. To instruct ArcGIS to log information, and where to write the logs, you need to set certain system environment variables.
SDEINTERCEPT says what to log.
SDEINTERCEPTLOC says where to log.
Client side: enabled by setting the environment variables before running the application.
Server side (three-tier connections only): enable by adding definitions for the two environment variables to the SDEHOME\etc\dbinit.sde and restarting the ArcSDE service. Caution: this will log all commands for all connected users. Do not leave this on, because there is some overhead in processing time and disk space

16. Queried tables & columns
SDEINTERCEPT Output
========================================
[W 14:30:54.582] Command: QueryWithInfo
[W 14:30:54.582] Long:
[W 14:30:54.582] Query Info:
Num Columns: 1
Columns: "Shape"
SQL_Construct: [1]
Tables: "scratch.MAP.STATES"
WhereClause: "STATE_NAME LIKE 'C%'"
Query Type: 4
ByClause: <null>
Num Hints: 0
Num Parameter markers: 0
Logfile: <null>
[R 14:30:54.582] Long:
[R 14:30:54.582] Col_Defines: [1]
Command name
Queried tables & columns
WHERE & ORDER BY
Clauses, query type
The SDEINTERCEPT log file includes commands and responses from ArcSDE server. Each item has a time stamp, preceded by and “R” (read) and “W” (write) to indicate the direction of the information flow.
Breaking down the QueryWithInfo command, it consists of these parts.
The command name.
The table and columns to include in the query.
Any WHERE or ORDER BY clauses to apply, if any.
The query type, which helps the ArcSDE server determine how to optimize the query. The query type designates whether the business table, feature table or spatial index tables should be joined, and whether the attribute or spatial portion of the query should be executed first. For more information, see the documentation on the C-API function SE_queryinfo_set_query_type.
The response, describing the columns that will be returned when the query is executed later. This information is needed by the client to establish appropriate storage for receiving the data from each column. Success from this command simply means that it was understood by the ArcSDE server. It does not mean that the query will ultimately succeed.
Response: Description
of columns returned

17. Additional Troubleshooting Resources

18. Monitoring Tools SDEINTERCEPT SDETRACE ExecuteSQL Database SQL Traces
Monitors commands generated by the ArcSDE client library
SDETRACE
Logs the calls from the client application to the ArcSDE C-API
ExecuteSQL
Control database SQL Traces from the ArcGIS client
Database SQL Traces
Records SQL received by the database engine, and what processing occurred in response to each request.
SDEINTERCEPT
When enabled, the ArcSDE built-in capability called SDEINTERCEPT monitors the commands that are generated by the ArcSDE client library for processing by the ArcSDE server.
These commands are not SQL. They are database-agnostic commands which correspond to specific requests to ArcSDE from the client.
SDEINTERCEPT records these commands, translating into human-readable form and placing in a text file.
SDETRACE
logs the calls from the client application to the ArcSDE C-API, providing a view of the interface between the client application code and the C-API.
The log file contains the name of each function in the order called, the time the call was made and when it completed.
This is useful to developers building applications and profiling its operation, or others familiar with the ArcSDE C-API who are diagnosing performance problems.
ExecuteSQL
This command may be used to executeSQL against a connected database. It uses the ArcObjects IWorkspace.ExecuteSQL method. It is very useful in starting DBMS tracing from ArcMap.
Database SQL Traces
- Commonly referred to as SQL traces, the capability to monitor the command execution of one or more session (connection) is provided by the database vendors.
Oracle: SQL traces ->TKPROF, or Enterprise Manager
SQL Server: The SQL Server Profiler
PostgreSQL: pglog -> pgfouine
Informix: sqlidebug
Major uses:
Show the SQL that was received by the database engine, and what processing occurred in response to each request.
Useful for looking at raw database errors before they are returned to ArcSDE.
Can trace some database events not directly caused by SQL commands can be logged (e.g. connections, DLL loading)
The level of detail of the trace is configurable.

19. Diagnostic Tools ArcSDE Error Messages SDEVERBOSE
ArcSDE errors/messages received on the client or in the logs
SDEVERBOSE
Logs errors/messages to sde_<instance>.log
Geodatabase Toolset (GDBT)
Monitoring, investigating and reporting
Replica Activity Logs
Records information for replica creation / synchronization
sdegdbrepair
Identifies and repairs any inconsistencies
ArcSDE Error Messages
ArcSDE errors/messages received on the client or in the logs.
Error messages you see when working with ArcSDE geodatabases can come from the underlying database management system, ArcSDE, or the client application. The following documentation link provides a list of some of the messages you might see returned from ArcSDE in the client interface or log files.
SDEVERBOSE
SDEVERBOSE reports internal messaging to the screen upon startup and writes gsrvr process startup and shutdown messages to sde_<instance>.log.
set SDEVERBOSE=TRUE is added to your dbinit.sde file in the SDEHOME\etc directory and should be removed when not in use.
The sde_<instance>.log is truncated each time the ArcSDE service is started.
How this can be used when troubleshooting a performance issue:
Spatial queries executed by each client are written to the ArcSDE instance's sde_<instance>.log. This allows the administrator or user to extract the SQL statement and execute the same query using SQL*Plus, which can help identify performance or resource contention issues.
FAQ:  What is the meaning of the 'Search SQL:' statement in the ArcSDE error log with Oracle?
Geodatabase Toolset (GDBT) for ArcCatalog
Tools that can assist you in monitoring, investigating and reporting the performance of you multiuser geodatabase.
Works with geodatabases stored on SQL Server, Oracle, Informix, DB2, or PostgreSQL databases.
Information provided through using the toolset:
The total number of rows in the add and delete tables
Users connected to the geodatabase and locked schemas
A graphic of the state tree lineage
Parent-child relationships between versions for version management
Detailed information of a feature class spatial index
Table and index statistics for geodatabases stored in an Oracle database
Currently available for 9.3 back to 9.1
NOTE: For viewing purposes only. Do not update the repository with the GDB Tools.
Replica Activity Logs
Records information for replica creation / synchronization:
ERRORS — How many errors occurred while running a process
WARNINGS — How many warnings occurred while running a process
Operation Name — The name of the process that was run
Time Completed — The date and time when the process finished
Operation Info — General information about the process
The log file is called ReplicaLog.dat and is found in the temp directory of the machine on which the operation was executed. If using ArcGIS Server, the information is stored in the Server Activity Log and can be accessed through the ArcGIS Server Manager.
HowTo:  Get a formatted view of the contents of the ReplicaLog.dat file
SDEGDBREPAIR
When to use?
The sdegdbrepair command is used to identify and repair any inconsistencies between the adds (A), the deletes (D) tables and versioning system tables of a versioned geodatabase.
The diagnose_tables and diagnose_metadata operations are used to determine if your database contains orphaned or duplicate rows in the delta tables or versioned business table.
The repair_tables and repair_metadata operations are used repair any inconsistencies identified in the delta tables or versioned business table.
Results are returned on screen but are also written to the sde_repair.log located in the SDEHOME\etc directory.

20. Online Resources ArcGIS Resource Center ArcGIS Web-based Help Forums
ArcGIS Web-based Help
Forums
Blogs