Technical Workshops | Esri International User Conference San Diego, California Managing Imagery and Raster Data Using Mosaic Datasets Hong Xu, Prashant Mangtani July 25, 2012

Presentation Overview Introduction to mosaic dataset Building mosaic datasets Design mosaic datasets for your organization Maintain and update Migrate to mosaic datasets Q&A

Challenges of Managing Imagery Many image sources - Aerial photographs - Satellite imagery from many sensors - DEM and scanned maps - Analytical data High resolution and large volume Need a data model - Store efficiently - Search easily - Process fast - Be able to accessible to multiple clients

What is a Mosaic Dataset A geodatabase model designed for managing collections of images - Catalog, process, search, display, and update Stored as tables in geodatabase - Footprint table references source images - Boundary, raster type - Seamlines, color correction etc… Displayed as a composite layer in Map - Boundary - Footprint - Image Mosaic Dataset

Advantages of Mosaic Datasets Dynamic mosaicking - Seamless display at all levels - Overviews to speed up display Processing images on-the-fly - Using raster function - Reduce process time Manage large image collections Fast to create Support images of multiple sensor platforms Streamline update and maintain quality

Use Mosaic Dataset as a Catalog Selection and query - Add selected rasters to Map - Lock the selected rasters View raster and metadata Time aware

Use Mosaic Dataset as an Image Is a raster dataset Provide a seamless mosaic on the fly based on the mosaic rule - Mosaic method to sort the rasters - Closest to center - By attribute - Closest to nadir - North west (default) - Seamline - Mosaic operator to resolve the overlaps - First/Min/Max/Mean/Blend Export or clip to a raster dataset Use as an input to geoprocessing tool Closest to the center By attribute: cloud cover

Share Mosaic Dataset as an Image Service Provide mosaic dataset functionality to web users Access as a catalog - Select/download selected images - Time aware Access as a seamless mosaic REST, Web API, WCS, WMS Require image extension license Session: Publishing image services in ArcGIS (Room 28D) - Thursday 08:30 AM - 09:45 AM - Thursday 03:15 PM - 04:30 PM

Demo: Mosaic dataset basics

Building Mosaic Datasets Cataloging and searching properties Make mosaic dataset look good - seamless mosaic

Basic Workflow Create a mosaic dataset Add rasters - Build footprint - Build boundary Build overviews (optional) Mosaic dataset toolset Automate with model and python

Footprint table – internal raster catalog with more fields Parameters to set - Define spatial reference - Web Mercator or spatial reference used by your organization - Number of bands and pixel types - Product definition - Used for automatic band matching - Examples: RGB, RGBI, Landset, etc Creating a Mosaic Dataset Create Mosaic Dataset tool - Create footprint table schema OIDShapeRasterNameMinPSMaxPSLowPSHighPSCategory… 1Polygon P01.met Primary

Raster value contains a raster function (process definition) - References images on disk - Process on the fly - Contains metadata, statistics, thumbnails A raster type defines what to add - Understand sensor metadata - Construct the raster function - Add additional fields - Sensor, CloudCover, etc. Adding Rasters Add Rasters to Mosaic Dataset tool - Populate field values OIDShapeRasterNameMinPSMaxPSLowPSHighPSCategory… 1Polygon P01.met Primary 2Polygon P02.met Primary

Raster dataset - TIFF, Mrsid, JP2000, NITF, etc. Satellite images - Landsat 7 ETM+, Landsat 4-5 TM, Landsat 1-5 MSS - QuickBird, IKONOS, GeoEye-1, WorldView I and II - RapidEye, SPOT, Radarsat 2, FORMOSAT-2, KOMPSAT-2 Aerial images - Match-AT, ISAT, Applanix Supported Raster Types Define data types can be managed by mosaic dataset

Lidar and Terrain - LAS, LAS Dataset, and Terrain Military Formats - CIB, CADRG, DTED, HRE Services - Image service, cache map service, WCS, WMS Table - Add existing mosaic datasets by copying all the rows Supported Raster Types (Continue) Define data types can be managed by mosaic dataset

Use DEM orthorectifying sensor data when possible Use Gram-Schmidt pansharpen method Statistics - Do not calculate for processed data - Calculate for sensor data Build pyramids Adding from folder when possible - Required by synchronizing a mosaic dataset Define datum transform if needed (GP environment) Handle invalid spatial reference - Define for missing data - Overwrite wrong spatial reference Tips for Adding Rasters

Mosaic dataset contains rasters of different resolutions MinPS and MaxPS define the visibility of these rasters Calculated based on sources and overlaps - Default cell size range factor is 10 Can be set using field calculator Calculate Mosaic Dataset Item Visibility Using Calculate Cell Size Ranges tool OIDShapeRasterNameMinPSMaxPSLowPSHighPSCategory… 1Polygon P01.met Primary 2Polygon P02.met Primary 5Polygon filename1.tif Overviews 5m visible range m visible range m visible range  5-30

Demo: Mosaic dataset cataloging and search

Topics for Building Mosaic Datasets Cataloging and searching Make mosaic dataset look good – seamless mosaic

Support many functions Edit one item or the mosaic dataset - using raster function editor Edit multiple items - using raster function batch editor The new Edit Raster Function tool - For editing item (s) or mosaic dataset Refine Processing – Raster Functions Edit raster function for mosaic items and mosaic dataset

Build Footprint tool - By radiometry - By geometry Edit using Editor Import Mosaic Dataset Geometry tool Define NoData GP tool - Based on a value - Based on a range Handling the Image Background Use Build Footprints and Set NoData tools Use footprint Use NoData

Define the boundary of the mosaic dataset - Pixels outside the boundary will be clipped Build using Build Boundary tool - Calculated based on footprints Can be modified using Editor Import Mosaic Dataset Geometry tool Build Boundary

Statistics in multiple places - Source raster dataset - Mosaic dataset item (function raster) - Mosaic dataset Pyramids exist in source dataset Calculate Statistics tool - for mosaic dataset statistics Build Pyramids and Statistics Tool - For statistics of items and source raster dataset Statistics in Mosaic Datasets Use Build Pyramids and Statistics tool Source rasterFunction raster Mosaic dataset

Based on a calculated color surface - Single color - Color grid - First, second, third polynomial Based on an existing target raster Support excluded area Color Correcting Mosaic Dataset Use Color Balance Mosaic Dataset tool or Color Correction window Before After

Used in mosaiking rasters in a mosaic dataset Stored in the seamline table Used by Seamline mosaic method - Blend along the seamlines with a default width Building Seamlines Use Build Seamlines tool 1 2 Generate seamlines automatically - Radiometry - Geometry - Etc. Edit and import seamlines

Using Build Overviews tool Building Overviews OIDShapeRasterNameMinPSMaxPSLowPSHighPSCategory… 1Polygon P01.met Primary 2Polygon P02.met Primary 3Polygon filename1.tif Overviews 4Polygon filename2.tif Overviews Cell size=100 Resampled rasters referenced in the footprint table Calculated based on default - TIFF with JPEG compression - Size of 5120x5120 and factor of 3 - Use Define Overview tool to refine the parameters Used for fast display at all levels Add a raster dataset or a service as an overview

Recommended for publishing as image services Check raster Check footprint geometry Check function chain Check broken path Performance etc. Analyzing Mosaic Datasets Using Analyze Mosaic Dataset tool

Data must be added from folder mosaic datasets Update existing items - Any change in source images - Any change in rasters - Option to maintain the existing fields Add new rasters Remove items of broken source Updating and Synchronizing Mosaic Dataset Using Synchronize Mosaic Dataset GP tool

Moving Mosaic Dataset Move all - Copy the FGDB where the mosaic dataset resides - Copy the source and overview images - Use Repair dialog to repair the paths - Or Repair Mosaic Dataset Paths GP tool Extract a portion - Use Distributed Geodatabase toolbar - Create a new folder - Create a target File geodatabase - Run Extract - Copy/move the whole folder

Demo: Make mosaic look good

Design mosaic datasets within your organization

Source mosaic dataset - Create from adding source images Master mosaic dataset - Create from source mosaic datasets - Table type by copying rows - Raster Dataset type by adding the whole mosaic dataset Support update mechanism - Synchronize source mosaic if source images updated - Synchronize master mosaic if source mosaic is updated Source and Master Mosaic Datasets

Reference Mosaic Dataset References an external mosaic dataset or raster catalog Created by specifying - Definition query - Area of interest It read-only (except boundary) Provides multiple views of the source mosaic dataset Prevents from editing the source mosaic dataset Updates when source mosaic is changed Where Sensor = Landsat and Cloud <10% Boundary = My County Add NDVI processing

A function template register with image service - Support client request - Reduced number of services need to be published Created using Function Template Editor - A custom button - Or saved from a raster function Defined when publishing an image service Server Raster Function (new in 10.1) For publishing image services

Mosaic Dataset Design Consideration Create mosaic datasets with data of similar type Elevation Ortho images of same date QuickBird, IKONOS Landsat 5 or 7 Use master mosaic datasets if needed Create referenced mosaic datasets Use server raster function to reduce the number of services Mosaic Dataset to manage data Reference Mosaic Dataset to publish

Demo: Build reference mosaic datasets using raster functions and server raster functions

From image service definition (.ISDef) - Create a mosaic dataset - Add using Image Service Definition type From raster process definition - Add using Raster Process Definition type Migrating to Mosaic Datasets

Migrating to Mosaic Datasets (Continue) Unmanaged Managed From a raster catalog - Create a mosaic dataset - Use Table type to add - Raster datasets are re-added as function raster datasets - Find the source data and re-create the mosaic dataset

What is New for Mosaic Dataset in 10.1 Support more raster types - LAS, LASDataset, Terrain, Radarsat2, Kompsat, Formosat, HRE, WMS Many New tools - Analyze Mosaic Dataset - Edit Raster Function - Set Mosaic Dataset Properties - Alter Mosaic Dataset Schema New raster functions - Remap, Band Arithmetic, Attribute Table, Apparent Reflectance, Tasseled Cap Enhancements - Automatic seamline generation - Calculate footprint by geometry - Gram-Schmidt pansharpen method - Support classify and Unique Value renderers - Automatic band matching using production definition

Summary Mosaic dataset is an effective model to manage large image collections Workflow of building mosaic datasets Various tips of building good a looking mosaic dataset Using master mosaic dataset, reference mosaic dataset to manage data in your organization Migrate mosaic datasets from image service definitions

Additional Information Technical workshops: Introduction to Imagery and Raster Data in ArcGIS (Wed:8:30, Fri:9:00) Publishing Image Services in ArcGIS (Thur:8:30 and 3:15) Advanced Image Management using the Mosaic Dataset (Wed: 3:15) Additional resources: Imagery section at resources.arcgis.com Image Blog Fill evaluations online First Offering ID: 880 Second Offering ID: 990

Questions?