3/8/ Managing Indianas 2005 Orthophotography in a Geodatabase Anna Radue University Information Technology Services (UITS) Indiana University

3/8/ Serving Raster Data in a RDBMS Why use ArcSDE? Why use ArcSDE? Overview of system hardware Overview of system hardware Basic raster ArcSDE Concepts Basic raster ArcSDE Concepts Geodatabase rastersGeodatabase rasters Pyramid layersPyramid layers Raster compressionRaster compression Loading tips and challenges Loading tips and challenges

3/8/ Isnt MrSID enough? Good for small regions Good for small regions Not appropriate for massively large datasets Not appropriate for massively large datasets Does not support multi-use Does not support multi-use No-Data areas may be a problem for seamless viewing No-Data areas may be a problem for seamless viewing

3/8/ Why Put Images in a Database? Multiuser Access Multiuser Access Common Data Management Common Data Management Access to very large images Access to very large images Data Security Data Security Performance Performance Database

3/8/ ArcSDE ESRIs Advanced Spatial Data Server ESRIs Advanced Spatial Data Server Serves Spatial Data to Serves Spatial Data to ArcGIS Desktop applicationsArcGIS Desktop applications Internet ClientsInternet Clients Applications developed with ArcGIS Engine and ArcGIS ServerApplications developed with ArcGIS Engine and ArcGIS Server Supports terabytes of imagery Supports terabytes of imagery

3/8/ Research Database Complex (RDC) Supercomputing system dedicated to Oracle database use Supercomputing system dedicated to Oracle database use Managed by RAC Unit within UITS Managed by RAC Unit within UITS 3 Solaris Servers 3 Solaris Servers Located in Bloomington machine room Located in Bloomington machine room Attached EMC CLARiiON system – networked storage system with FC and ATA disks Attached EMC CLARiiON system – networked storage system with FC and ATA disks

3/8/ Solaris Servers Oracle & ArcIMS Spatial Server Oracle & ArcIMS Spatial Server Sun Fire V1280 ServerSun Fire V1280 Server 12 UltraSPARC III Processors 12 UltraSPARC III Processors 96 GB of memory 96 GB of memory Gb switched ethernet Gb switched ethernet Apache/Tomcat/ArcIMS Web Server Apache/Tomcat/ArcIMS Web Server Sun Fire V880 ServerSun Fire V880 Server 4 UltraSPARC III Processors 4 UltraSPARC III Processors 8 GB of memory 8 GB of memory

3/8/ Fiber-Channel Disks Servers share Servers share Provides the highest level of disk access performance and capacity available in the storage marketplace Provides the highest level of disk access performance and capacity available in the storage marketplace Technology supports peak concurrent client loads Technology supports peak concurrent client loads

3/8/ Parallel ATA Disks Advanced Technology Attachment (ATA) – std first introduced in 1986 Advanced Technology Attachment (ATA) – std first introduced in 1986 Generally relegated to non-server scenarios such as laptops and desktops Generally relegated to non-server scenarios such as laptops and desktops About 50% the cost of fibre channel disks About 50% the cost of fibre channel disks

3/8/ Disk Access Performance ESRI Systems Integration Technical Brief: Image Storage Architecture Prepared by: Dave Peters, ESRI

3/8/ EMC Storage System Two-Tier Architecture Combination of Fibre channel and parallel ATA disks in RAID system Combination of Fibre channel and parallel ATA disks in RAID system Oracle system tables, vector data, database index and log files on FC Oracle system tables, vector data, database index and log files on FC Raster imagery on ATA Raster imagery on ATA FC ATA

3/8/ Geodatabase Rasters 1. Raster Catalog 2. Raster Dataset 3. Hybrid Raster Catalog

3/8/ Raster Catalog Individual images are displayed as seamless imageIndividual images are displayed as seamless image Useful for users having frequent image updatesUseful for users having frequent image updates Maintains areas of overlapMaintains areas of overlap Image cells do not need to alignImage cells do not need to align Database can store the images or referenceDatabase can store the images or reference

3/8/ Raster Dataset A raster composed of multiple input images – 2813 for DOQQ 1998 A raster composed of multiple input images – 2813 for DOQQ 1998 Image overlap not kept Image overlap not kept Cells must align Cells must align Mosaiced rasters perform better than a raster catalog Mosaiced rasters perform better than a raster catalog

3/8/ Hybrid Raster Catalog ArcCatalog ArcMap

3/8/ What are pyramid layers? Reduced resolution representations of the data set Reduced resolution representations of the data set Improve display performance Improve display performance Increase size of image Increase size of image

3/8/ Image and 2 Pyramids

3/8/ Pyramid Stats

3/8/ Pyramid Resampling Methods Nearest Neighbor Nearest Neighbor Bilinear Bilinear BiCubic BiCubic

3/8/ Compression LZ77 LZ77 LosslessLossless Requires the most disk spaceRequires the most disk space JPEG JPEG Lossy – different qualitiesLossy – different qualities 75% ESRI default75% ESRI default JPEG 2000 JPEG 2000 lossy and lossless depending on qualitylossy and lossless depending on quality Loading and pyramid building SLOWLoading and pyramid building SLOW

3/8/ JPG 75%, bilinear pyramid

3/8/ ArcSDE Tips and Tricks #1 Create the Raster Catalog using ArcToolbox Utilities Create the Raster Catalog using ArcToolbox Utilities Unless loading a small number of images, load imagery using sderaster command – trying to load 200+ images in ArcCatalog 9.1 crashes the application even on robust systems Unless loading a small number of images, load imagery using sderaster command – trying to load 200+ images in ArcCatalog 9.1 crashes the application even on robust systems

3/8/ ArcSDE Tips and Tricks #2 Spatial Domain Change Spatial Domain default values Change Spatial Domain default values

3/8/ ArcSDE Tips and Tricks #3 Output Spatial Grid Set to 250,000

3/8/ ArcSDE Tips and Tricks #4 ArcSDE Views Views filter information stored in ArcSDE layers and database tables Views filter information stored in ArcSDE layers and database tables Simplifies or limits data access for the end user Simplifies or limits data access for the end user Access to information stored in database tables is controlled with where clauses – only completely loaded counties are viewable by end users – partially loaded counties without pyramids will always hang ArcMap Access to information stored in database tables is controlled with where clauses – only completely loaded counties are viewable by end users – partially loaded counties without pyramids will always hang ArcMap

3/8/ ArcSDE Raster Datasets Serve imagery quickly to clients even using lower performance disks Serve imagery quickly to clients even using lower performance disks Can store terabytes of imagery Can store terabytes of imagery Indiana Spatial Data ServiceIndiana Spatial Data Service Indiana Spatial Data PortalIndiana Spatial Data Portal

3/8/ Acknowledgements ArcSDE/Oracle Consultant Mannion Geosystems, LLC ArcSDE/Oracle Consultant Mannion Geosystems, LLC Data Management Support, UITS Data Management Support, UITS Michael Halla, ManagerMichael Halla, Manager Nathan EatonNathan Eaton Stephanie SniderStephanie Snider Research and Technical Services, UITS Research and Technical Services, UITS Stephanie Burks, Oracle AdministratorStephanie Burks, Oracle Administrator Teresa Hunter, UNIX AdministratorTeresa Hunter, UNIX Administrator

3/8/ Financial and Data Support IGIC FGDC CAP Grant purchased ATA disks Jill Saligoe-Simmel IGIC FGDC CAP Grant purchased ATA disks Jill Saligoe-Simmel Department of Homeland Security purchased FC disks and approved service Roger Koelpin Department of Homeland Security purchased FC disks and approved service Roger Koelpin Pinnacle providing IU with imagery Phil Worrall Pinnacle providing IU with imagery Phil Worrall

3/8/ Questions?