Computer Vision and Digital Photogrammetry Methodologies for Extracting Information and Knowledge from Remotely Sensed Data Toni Schenk, CEEGS Department, OSU Bea Csatho, Department of Geology, University at Buffalo (formerly with BPRC, OSU)

From Data to Knowledge data collection preprocessing raw data feature extraction data/information analysis knowledge data specialist geo scientist experience diagram effort diagram of geoscientist

From Data to Information and Knowledge data collection preprocessing raw data information extraction information analysis knowledge data special ist geo scientist experience diagrams effort diagram of geoscientist geospatial information provider

From Data to Information and Knowledge in Human Vision scene information knowledge data perception data acquisition

Paradigm data acquisition preprocessing raw data feature extraction perceptual organization applications datainformation knowledge

Data, Information, Knowledge data information knowledge representation semantic content signalsymbolic

Data sensor output, result of data acquisition raw signals or preprocessed data is not an end product, nor does it directly provide answers to applications examples: pixels, laser points  data needs to be (further) processed

Information more specific than data to answer questions information can be retrieved from data  information implicitly in data extracting information from data makes it explicit  feature extraction features are information primitives

Knowledge elusive concept, ill-defined  means different things to different people to ‘deal’ with knowledge it must be represented  knowledge and its representation are closely related knowledge: facts, procedures, heuristics that can be used to make inferences

Example of feature extraction and their perceptual organization features are likely to correspond to object boundaries (pixels carry no explicit information about scene) perceptual organization of features leads to structures that can be used to generate hypotheses about objects

Test Site Test Site Ocean CityDetail View

Edge Detection

Edge Segmentation segmentation of raw edges into straight lines threshold criteria straightness length from pixels to vectors

Grouping: Rectangular Edges and Corners

Edge Information segmentation and grouping are perceptual processes perceptual organization transforms data (edge pixels) into information (parallel edges, rectangular edges, corners,…) edge information enters spatial reasoning process

From Data to Knowledge in a Multisensor Scenario data information knowledge query

Information and Knowledge Generating Processes

Feature Extraction From Different Sensors aerial/satellite imagery:  edges by edge operators  regions, e.g. by texture segmentation LIDAR:  smooth surface patches, e.g. planar and higher order surfaces multispectral/hyperspectral imagery:  regions with similar spectral properties

Perceptual Organization of Laser Point Clouds find structure in randomly distributed 3-D point clouds feature extraction, e.g. planar surface patches segmentation, merging grouping: establish topological relationships, bare-earth separation, compute higher order features

Fusion of Aerial Imagery and Lidar: Data Level relationship? imageLidar

Fusion of Aerial Imagery and Lidar: Extracted Features, 1 st Level relationship? Lidar region 1region 2 image raw edge

Fusion of Aerial Imagery and Lidar: Extracted Features, 2 nd Level relationship = same physical phenomena region 2region 1 Lidar computed edge image segmented edge sensor invariant feature

aerial image Principle of Image Registration laser points visible surface X Y Z reference frame aerial image

ICESat Laser Points Back-Projected To Stereo Images: Before Registration

ICESat Laser Points Back-Projected To Stereo Images: After Registration

Concluding Remarks process of extracting useful information improves the ratio data gathering/knowledge creation more data can be ‘mined’ ‘useful’ information is application dependent

Concluding Remarks geoinformatics is multi-disciplinary  sensor designers, mission planners  data collection specialists  geoinformation providers photogrammetry, remote sensing, computer vision, computer science  geoinformation managers (GIS)  geoscientists