1. CATENA Distributed Generic Processing Chain for Optical Satellite Imagery Processing
Peter Reinartz, Thomas Krauß
Remote Sensing Technology Institute Photogrammetry and Image Analysis
ESA Workshop on Models for Scientific Exploitation of EO data Frascati,

2. Why processing chains for higher level optical data processing?
Needed for:
Processing of large areas and large data volumes e.g. Image2006/2009/2012, each time about 3500 scenes IRS/SPOT for whole Europe-38
Processing of time series e.g. CCI-Fire, Meris/ATSR/SPOT-VGT for , about scenes
Requirements:
Fully automatic processing of
Mass data from
Many optical sensors/satellites
Modular and easy re-configurable for many projects
Image 2006, ~3500 IRS/SPOT scenes

3. Atmospheric correction
CATENA
CATENA – chain for fully automatic processing of optical satellite data
Automatic and operational processing chain for processing of mass data
Using global databases and reference data
Support of native satellite image formats from SPOT4/5, IRS-P6 Liss3/AWiFS, ALOS AVNIR/PRISM, Ikonos, Quickbird, RapidEye, WorldView, GeoEye, Cartosat, Pleiades, Meris, ATSR, VGT, Modis, …
Reference image
DEM
input
output
Ortho image
Original image
Image
Matching
Sensor Model
Refinement
Ortho- rectification
Atmospheric correction
extract ground control points from global
Reference databases
perform
parameter
estimation
Use global
DEM database
remove
atmospheric
influence
Thematic processing

4. CATENA CATENA – Use cases Catena is A chain of processing modules
Uses interface- and data standards
Usable as DIMS- or stand-alone-version
Example use cases are
Orthorectification in Image2006, Image2009, UrbanAtlas, ...
DEM-Generation as service for Cartosat/Euromap
Stereoprocessing, Time series, CCI-Fire, ...

5. CATENA CATENA – Requirements Systems and Libraries:
Linux (tested on CentOS, Ubuntu)
XDibias (DLR in-house development)
Python, scipy, numpy
GDAL
Modules:
Must not be interactive (automatic processing chain!)
Preferable: UNIX C/C++ source code, python code
Possible: Java, Fortran, any standard UNIX (script) programming language
No commercial programming environments which require any kind of licenses!

6. CATENA CATENA – Interfaces Input-Data:
Original Level-1-satellite data containing all metadata
Processed data including required metadata
Modules:
Image data and metadata in standardized XDibias format
Modules wrap existing processors with configuration files and any image format supported by GDAL
Output:
Any image format supported by GDAL
Standardized export.xml containing meta- and processing info
JPG-Quicklooks, KML files, any other intermediate files

7. CATENA – Summary of Principles
Standardized image- and metadata formats
Standardized process flow organization
Processor follows ESA „Generic IPF Interface Specifications“
Distributed computing and storage
Standardized Development and Deployment process
Guidelines for module development, documentation and deployment
ISO9000 certification in process: external audit today ( )

8. CATENA – System overview
Modules and order defined in chain
Some modules need additional data
Select processing chain and set parameters
Original data
Import
Processing chains
Reference DB 1
Module 1
Module 2
Ingestion
Reference DB 2
Module 3
Ortho
DEM-Generation
Atmospheric Corr.
CCI-Fire
Chain . . .
Delivery
Module 4
Reference DB 3
Delivered data
Module ...
Export
Standardized image and meta data
Work- space
Processing control system
Web-Interface
Each job gets processed in own space
Cleaned up after delivery
DIMS-PSM or stand-alone

9. CATENA – Grid computing
Reference data
crontab
crontab
work
work
Node 2
crontab
data
Node 1
work
crontab
crontab
Node 3
. . .
work
work
Node 5
Node 4
crontab
crontab
work
work
scene database DB
crontab
work
crontab
Server
work
Simply add new node by creating working directory and inserting CATENA into crontab
Node 6
Node n
web server

10. CATENA – Distributed Mass Storage
Distributed mass storage with access from each processing node is needed for automatic processing of time series or bulk data for:
Realized as easily expansible Scality storage ring:
Data is stored automatically in three distributed copies in the ring, read-access also in parallel from three storage nodes.
Data

11. Web-Interface of stand-alone version
CATENA
Web-Interface of stand-alone version

12. CATENA
Examples
Orthorectification geocoded, optionally atmospheric corrected satellite images for further thematic processing and emergency mapping
DEM generation generate DEMs and Ortho images from (multi) stereo satellite data

13. Processing chain: Orthorectification
CATENA
Processing chain: Orthorectification
Standard processing chain for most optical satellite data
Satellites acquire oblique images
Ephemeris and attitude not exactly known
Correct these using ground control points
from already existing geocoded images
Project satellite image on existing digital elevation model
from DEM database (e.g. SRTM)
Resample satellite image in requested projection and resolution

14. Processing chain: Orthorectification Workflow
CATENA
Processing chain: Orthorectification Workflow
Original image
Reference image
Matching
DEM
Control points
Improvement of orbit and attitude data
Manually measured
ground control points
Generation of ortho image
Quality check
Delivery
Probleme bei Qualität z.B. ungünstige Bewölkung, starke Änderungen von Referenz zu neuem Bild
Prozessor generell auch für Flugzeugdaten geeignet
BW: Diese Übersicht beschreibt auch gleichzeitig den Ablauf wie wir ihn für die SAR-GTC-Prozessierung anstreben.
AP420 einzelne, neue Modulentwicklungen (Thema Passpunkte), AP430 Prozessketten
Ortho image
Atmospheric Correction
Thematic processing

15. Overall Geometric Accuracy
Requirement: RMSE < 20m
Overall (~4000 scenes) mean accuracy w.r.t. reference data set:
RMSEx/y ~ 10 m ( CE64 ~14m)
~0.5 pixel size of resampled images
Coverage 1
Coverage 2
RMSE X Y X Y
Mean number of ICPs per scene for accuracy assessment:
IRS-P6: points / scene
SPOT 4/5: points / scene
Residual plots available

16. Processing chain: DEM generation Workflow
Metadata
Orientation
Matching
DEM Generation
Ortho- rectifi- cation
Images
DEM
Ortho
Input
Processing
Output
At least two images from same orbit
Good relative orientation required, <0.5px, Bundle block adjustment
Dense pixelwise Semi-Global Matching = Disparity map on original images
Reprojection of DEM to target coordiante system, Interpolation and filling of holes
Orthorectification of the original imagery

17. Processing chain: DEM generation Ortho image and DEM, London

18. Processing chain: DEM generation London DSM from 5 WorldView-2 Images

19. Processing chain: DEM generation K2 WorldView-2 Triple Stereo
Processing Chains for Optical Data • Thomas Krauß • • • Slide 19
Processing chain: DEM generation K2 WorldView-2 Triple Stereo
15°
Very steep terrain
Very detailed surface model
Film: 0°
-15°

20. Processing Chains for Optical Data Summary
CATENA
Processing Chains for Optical Data Summary
Processing chain CATENA developed at the Remote Sensing Technology Institute of DLR for fully automatic processing of mass data from many different optical satellites
Already in use for many projects (Image , UrbanAtlas, CCI-Fire, Cartosat-DEM-processor, Worldview-2 and Pleiades DEM generation, …)
Based on the general processing chain infrastructure CATENA including:
Modular system of processing Modules connected to Chains
Distributed parallel grid computing
Distributed mass storage
Easily expandable, e.g.:
A new processing chain for a new project
Adding normal Linux-PCs or virtual machines as new background processing nodes
Contact: Thomas Krauß, DLR-IMF,

21. Thank you for your attention