1. 3/8/2006 http://gis.iu.edu 1 Managing Indianas 2005 Orthophotography in a Geodatabase Anna Radue University Information Technology Services (UITS) Indiana University

2. 3/8/2006 http://gis.iu.edu 2 Serving Raster Data in a RDBMS Why use ArcSDE? Why use ArcSDE? Overview of system hardware Overview of system hardware Basic raster ArcSDE Concepts Basic raster ArcSDE Concepts Geodatabase rastersGeodatabase rasters Pyramid layersPyramid layers Raster compressionRaster compression Loading tips and challenges Loading tips and challenges

3. 3/8/2006 http://gis.iu.edu 3 Isnt MrSID enough? Good for small regions Good for small regions Not appropriate for massively large datasets Not appropriate for massively large datasets Does not support multi-use Does not support multi-use No-Data areas may be a problem for seamless viewing No-Data areas may be a problem for seamless viewing

4. 3/8/2006 http://gis.iu.edu 4 Why Put Images in a Database? Multiuser Access Multiuser Access Common Data Management Common Data Management Access to very large images Access to very large images Data Security Data Security Performance Performance Database

5. 3/8/2006 http://gis.iu.edu 5 ArcSDE ESRIs Advanced Spatial Data Server ESRIs Advanced Spatial Data Server Serves Spatial Data to Serves Spatial Data to ArcGIS Desktop applicationsArcGIS Desktop applications Internet ClientsInternet Clients Applications developed with ArcGIS Engine and ArcGIS ServerApplications developed with ArcGIS Engine and ArcGIS Server Supports terabytes of imagery Supports terabytes of imagery

6. 3/8/2006 http://gis.iu.edu 6 Research Database Complex (RDC) Supercomputing system dedicated to Oracle database use Supercomputing system dedicated to Oracle database use Managed by RAC Unit within UITS Managed by RAC Unit within UITS 3 Solaris Servers 3 Solaris Servers Located in Bloomington machine room Located in Bloomington machine room Attached EMC CLARiiON system – networked storage system with FC and ATA disks Attached EMC CLARiiON system – networked storage system with FC and ATA disks

7. 3/8/2006 http://gis.iu.edu 7 Solaris Servers Oracle & ArcIMS Spatial Server Oracle & ArcIMS Spatial Server Sun Fire V1280 ServerSun Fire V1280 Server 12 UltraSPARC III Processors 12 UltraSPARC III Processors 96 GB of memory 96 GB of memory Gb switched ethernet Gb switched ethernet Apache/Tomcat/ArcIMS Web Server Apache/Tomcat/ArcIMS Web Server Sun Fire V880 ServerSun Fire V880 Server 4 UltraSPARC III Processors 4 UltraSPARC III Processors 8 GB of memory 8 GB of memory

8. 3/8/2006 http://gis.iu.edu 8 Fiber-Channel Disks Servers share Servers share Provides the highest level of disk access performance and capacity available in the storage marketplace Provides the highest level of disk access performance and capacity available in the storage marketplace Technology supports peak concurrent client loads Technology supports peak concurrent client loads

9. 3/8/2006 http://gis.iu.edu 9 Parallel ATA Disks Advanced Technology Attachment (ATA) – std first introduced in 1986 Advanced Technology Attachment (ATA) – std first introduced in 1986 Generally relegated to non-server scenarios such as laptops and desktops Generally relegated to non-server scenarios such as laptops and desktops About 50% the cost of fibre channel disks About 50% the cost of fibre channel disks

10. 3/8/2006 http://gis.iu.edu 10 Disk Access Performance ESRI Systems Integration Technical Brief: Image Storage Architecture Prepared by: Dave Peters, ESRI

11. 3/8/2006 http://gis.iu.edu 11 EMC Storage System Two-Tier Architecture Combination of Fibre channel and parallel ATA disks in RAID system Combination of Fibre channel and parallel ATA disks in RAID system Oracle system tables, vector data, database index and log files on FC Oracle system tables, vector data, database index and log files on FC Raster imagery on ATA Raster imagery on ATA FC ATA

12. 3/8/2006 http://gis.iu.edu 12 Geodatabase Rasters 1. Raster Catalog 2. Raster Dataset 3. Hybrid Raster Catalog

13. 3/8/2006 http://gis.iu.edu 13 Raster Catalog Individual images are displayed as seamless imageIndividual images are displayed as seamless image Useful for users having frequent image updatesUseful for users having frequent image updates Maintains areas of overlapMaintains areas of overlap Image cells do not need to alignImage cells do not need to align Database can store the images or referenceDatabase can store the images or reference

14. 3/8/2006 http://gis.iu.edu 14 Raster Dataset A raster composed of multiple input images – 2813 for DOQQ 1998 A raster composed of multiple input images – 2813 for DOQQ 1998 Image overlap not kept Image overlap not kept Cells must align Cells must align Mosaiced rasters perform better than a raster catalog Mosaiced rasters perform better than a raster catalog

15. 3/8/2006 http://gis.iu.edu 15 Hybrid Raster Catalog ArcCatalog ArcMap

16. 3/8/2006 http://gis.iu.edu 16 What are pyramid layers? Reduced resolution representations of the data set Reduced resolution representations of the data set Improve display performance Improve display performance Increase size of image Increase size of image

17. 3/8/2006 http://gis.iu.edu 17 Image and 2 Pyramids

18. 3/8/2006 http://gis.iu.edu 18 Pyramid Stats

19. 3/8/2006 http://gis.iu.edu 19 Pyramid Resampling Methods Nearest Neighbor Nearest Neighbor Bilinear Bilinear BiCubic BiCubic

20. 3/8/2006 http://gis.iu.edu 20 Compression LZ77 LZ77 LosslessLossless Requires the most disk spaceRequires the most disk space JPEG JPEG Lossy – different qualitiesLossy – different qualities 75% ESRI default75% ESRI default JPEG 2000 JPEG 2000 lossy and lossless depending on qualitylossy and lossless depending on quality Loading and pyramid building SLOWLoading and pyramid building SLOW

21. 3/8/2006 http://gis.iu.edu 21 JPG 75%, bilinear pyramid

22. 3/8/2006 http://gis.iu.edu 22 ArcSDE Tips and Tricks #1 Create the Raster Catalog using ArcToolbox Utilities Create the Raster Catalog using ArcToolbox Utilities Unless loading a small number of images, load imagery using sderaster command – trying to load 200+ images in ArcCatalog 9.1 crashes the application even on robust systems Unless loading a small number of images, load imagery using sderaster command – trying to load 200+ images in ArcCatalog 9.1 crashes the application even on robust systems

23. 3/8/2006 http://gis.iu.edu 23 ArcSDE Tips and Tricks #2 Spatial Domain Change Spatial Domain default values Change Spatial Domain default values

24. 3/8/2006 http://gis.iu.edu 24 ArcSDE Tips and Tricks #3 Output Spatial Grid Set to 250,000

25. 3/8/2006 http://gis.iu.edu 25 ArcSDE Tips and Tricks #4 ArcSDE Views Views filter information stored in ArcSDE layers and database tables Views filter information stored in ArcSDE layers and database tables Simplifies or limits data access for the end user Simplifies or limits data access for the end user Access to information stored in database tables is controlled with where clauses – only completely loaded counties are viewable by end users – partially loaded counties without pyramids will always hang ArcMap Access to information stored in database tables is controlled with where clauses – only completely loaded counties are viewable by end users – partially loaded counties without pyramids will always hang ArcMap

26. 3/8/2006 http://gis.iu.edu 26 ArcSDE Raster Datasets Serve imagery quickly to clients even using lower performance disks Serve imagery quickly to clients even using lower performance disks Can store terabytes of imagery Can store terabytes of imagery http://gis.iu.edu http://gis.iu.edu Indiana Spatial Data ServiceIndiana Spatial Data Service Indiana Spatial Data PortalIndiana Spatial Data Portal

27. 3/8/2006 http://gis.iu.edu 27 Acknowledgements ArcSDE/Oracle Consultant Mannion Geosystems, LLC ArcSDE/Oracle Consultant Mannion Geosystems, LLC Data Management Support, UITS Data Management Support, UITS Michael Halla, ManagerMichael Halla, Manager Nathan EatonNathan Eaton Stephanie SniderStephanie Snider Research and Technical Services, UITS Research and Technical Services, UITS Stephanie Burks, Oracle AdministratorStephanie Burks, Oracle Administrator Teresa Hunter, UNIX AdministratorTeresa Hunter, UNIX Administrator

28. 3/8/2006 http://gis.iu.edu 28 Financial and Data Support IGIC FGDC CAP Grant purchased ATA disks Jill Saligoe-Simmel IGIC FGDC CAP Grant purchased ATA disks Jill Saligoe-Simmel Department of Homeland Security purchased FC disks and approved service Roger Koelpin Department of Homeland Security purchased FC disks and approved service Roger Koelpin Pinnacle providing IU with imagery Phil Worrall Pinnacle providing IU with imagery Phil Worrall

29. 3/8/2006 http://gis.iu.edu 29 Questions? aradue@indiana.edu