Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035.

Slides:



Advertisements
Similar presentations
Artificial Neural Networks
Advertisements

Memristor in Learning Neural Networks
A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.
NEURAL NETWORKS Backpropagation Algorithm
1 Image Classification MSc Image Processing Assignment March 2003.
Artificial Intelligence 13. Multi-Layer ANNs Course V231 Department of Computing Imperial College © Simon Colton.
Neural Networks: Backpropagation algorithm Data Mining and Semantic Web University of Belgrade School of Electrical Engineering Chair of Computer Engineering.
Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.
Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management University of California, Berkeley May 18, 2005 Classifying.
ASTER image – one of the fastest changing places in the U.S. Where??
Machine Learning Neural Networks
Multiple Criteria for Evaluating Land Cover Classification Algorithms Summary of a paper by R.S. DeFries and Jonathan Cheung-Wai Chan April, 2000 Remote.
Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Neural Networks. R & G Chapter Feed-Forward Neural Networks otherwise known as The Multi-layer Perceptron or The Back-Propagation Neural Network.
Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Neural Networks Chapter Feed-Forward Neural Networks.
Artificial Neural Networks (ANNs)
Classification of Remotely Sensed Data General Classification Concepts Unsupervised Classifications.
CHAPTER 11 Back-Propagation Ming-Feng Yeh.
SOMTIME: AN ARTIFICIAL NEURAL NETWORK FOR TOPOLOGICAL AND TEMPORAL CORRELATION FOR SPATIOTEMPORAL PATTERN LEARNING.
Artificial Intelligence (AI) Addition to the lecture 11.
An Object-oriented Classification Approach for Analyzing and Characterizing Urban Landscape at the Parcel Level Weiqi Zhou, Austin Troy& Morgan Grove University.
Image Classification and its Applications
Artificial Neural Networks
Artificial Neural Nets and AI Connectionism Sub symbolic reasoning.
1 Urban Growth Simulation A Case Study of Indianapolis Sharaf Alkheder & Jie Shan School of Civil Engineering Purdue University March 10, 2005.
© Copyright 2004 ECE, UM-Rolla. All rights reserved A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C.
11 CSE 4705 Artificial Intelligence Jinbo Bi Department of Computer Science & Engineering
Image Classification Digital Image Processing Techniques Image Restoration Image Enhancement Image Classification Image Classification.
Machine Learning Dr. Shazzad Hosain Department of EECS North South Universtiy
NEURAL NETWORKS FOR DATA MINING
LINEAR CLASSIFICATION. Biological inspirations  Some numbers…  The human brain contains about 10 billion nerve cells ( neurons )  Each neuron is connected.
Image Classification 영상분류
ARTIFICIAL NEURAL NETWORKS. Overview EdGeneral concepts Areej:Learning and Training Wesley:Limitations and optimization of ANNs Cora:Applications and.
Artificial Neural Networks An Introduction. What is a Neural Network? A human Brain A porpoise brain The brain in a living creature A computer program.
What is an image? What is an image and which image bands are “best” for visual interpretation?
Land Cover Change Monitoring change over time Ned Horning Director of Applied Biodiversity Informatics
Land Cover Change Monitoring change over time Ned Horning Director of Applied Biodiversity Informatics
Digital Image Processing
Application of spatial autocorrelation analysis in determining optimal classification method and detecting land cover change from remotely sensed data.
Back-Propagation Algorithm AN INTRODUCTION TO LEARNING INTERNAL REPRESENTATIONS BY ERROR PROPAGATION Presented by: Kunal Parmar UHID:
Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.
Neural Network and Deep Learning 王强昌 MLA lab.
Artificial Intelligence Methods Neural Networks Lecture 3 Rakesh K. Bissoondeeal Rakesh K. Bissoondeeal.
BACKPROPAGATION (CONTINUED) Hidden unit transfer function usually sigmoid (s-shaped), a smooth curve. Limits the output (activation) unit between 0..1.
CHANGE DETECTION ANALYSIS USING REMOTE SENSING TECHNIQUES Change in Urban area from 1992 to 2001 in COIMBATORE, INDIA. FNRM 5262 FINAL PROJECT PRESENTATION.
Kim HS Introduction considering that the amount of MRI data to analyze in present-day clinical trials is often on the order of hundreds or.
Machine Learning Artificial Neural Networks MPλ ∀ Stergiou Theodoros 1.
A Presentation on Adaptive Neuro-Fuzzy Inference System using Particle Swarm Optimization and it’s Application By Sumanta Kundu (En.R.No.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining.
Big data classification using neural network
Deep Learning Amin Sobhani.
HIERARCHICAL CLASSIFICATION OF DIFFERENT CROPS USING
Classification of Remotely Sensed Data
ASTER image – one of the fastest changing places in the U.S. Where??
Incorporating Ancillary Data for Classification
Artificial Neural Networks
By Yudhi Gunawan * and Tamás János **
Evaluating Land-Use Classification Methodology Using Landsat Imagery
Self organizing networks
CSC 578 Neural Networks and Deep Learning
Chapter 3. Artificial Neural Networks - Introduction -
Neuro-Computing Lecture 4 Radial Basis Function Network
Unsupervised Classification
Image Information Extraction
network of simple neuron-like computing elements
Zip Codes and Neural Networks: Machine Learning for
Seminar on Machine Learning Rada Mihalcea
Outline Announcement Neural networks Perceptrons - continued
Presentation transcript:

page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) /07/2002

page 2 of 50 Outline Introduction of Satellite Remote Sensing Remote Sensing Image Classification Methodology Data and Experiments Discussion and Conclusion

page 3 of 50 Earth Surface and Land Cover Map How to achieve an accurate land cover map in natural resources research?

page 4 of 50 Remote Sensing Remote Sensing is defined as: The science of acquiring, processing and interpreting images that record the interaction between electromagnetic energy and matter.

page 5 of 50 Different Type of Land Use and Land cover and Spectral Characteristics Blue Band Red BandNear-infrared Band Green Band

page 6 of 50 From Digital Image to Land-use Map Visualization Classification Multi-spectral Digital Image Pseudo Color Image Land-use and Land-cover Map

page 7 of 50 Classification System In this study, we used the USGS classification with 10 land cover categories: 1. Turf/Grass 2. Barren land 3. Conifer forest 4. Deciduous forest 5. Mixed forest 6. Brush land 7. Urban land 8. Water 9. Non-forest wetland 10. Forest wetland

page 8 of 50 Remote Sensing Image Classification Statistical Methods – Supervised – Unsupervised Artificial Neural Network Approaches Rule-based...

page 9 of 50 Supervised Remote Sensing Image Classification Vegetation Water Soil Band 1 (0 ~ 255) Band 2 (0 ~ 255)

page 10 of 50 Classification Process Mapping Relationship Methods: Statistical classifier ANN-based classifier Observation space Solution space Landsat TM Band1 Band2 Band3 Band4 Band5 Band6 Band7 0~255 0~255 0~255 0~255 0~255 0~255 0~255 Category 1 Category.. Category … Category N Water wetland Forest Agri. Urban Residential Category: Forest (Pattern)

page 11 of 50 Statistical Methods - Need Gaussian (Normal) distribution on the input data which is required by Bayesian classifier. - Restrictions about the format of input data. Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Water Soil

page 12 of 50 Statistical Methods: Deciduous forest Conifer forest Forest Wetland Mixed forest Brush Land Non-forest Wetland How to find a boundary for the following patterns?

page 13 of 50 Artificial Neural Network Approach No need for normal distribution on input data Flexibility on input data format Improved classification accuracy Robust and reliability

page 14 of 50 ………………………… …………………… Artificial Neural Network Is Defined by... Processing elements Organized topological structure Learning rules

page 15 of 50 Processing Element (PE)  f(x) Output Input PE Artificial counterparts of neurons in a brain W j1 W j2 W j4 W j3 W j5

page 16 of 50 PE’s Output Function of Processing Elements ………………………… …………………… unit j o1o1 o2o2 o3o3 f w j1 w j2 w j3 ojoj PE’s Inputs Receive outputs from each PEs locate in previous layer. Compute the output with a Sigmoid activation function F( Sumof( O i * W ji) ) Transfer the output to all the PEs in next layer

page 17 of 50 Input layerHidden layerOutput layer ………………………… …………………… Input vector i(x 1, x 2, … x n ) Output vector i(o 1, o 2, … o m ) Artificial Neural Network Architecture - Backpropagation ANN (BPANN) Feed InformationAnalysis Result

page 18 of 50 Pattern Recognition Vegetation: (10, 89) > (1,0,0) (11,70) > (1,0,0) … … … (1,0,0) Water: (10, 21) > (0,1,0) (15, 32) > (0,1,0) … … … (0,1,0) Soil: (50, 40) > (0,0,1) (52, 40) > (0,0,1) … … … (0,0,1) Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Wa t er Soil

page 19 of 50 ANN Design Pattern Input layerHidden layerOutput layer …………………… How many PEs we need - Basic rules in designing an ANN. Input layer PEs - by dimension of input vector Output layer PEs - by total number of patterns (classes) …………………… Feed Forward Back-Propagate (0 ~ 255) (0 ~ 1) Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Water Soil

page 20 of 50 ANN Training - From Pattern to Land Cover Category Pattern …………………… …………………… Feed Forward Back-Propagate Vegetation: (10, 89) > (1,0,0) (11,70) > (1,0,0) … … … (1,0,0) Water: (10, 21) > (0,1,0) (15, 32) > (0,1,0) … … … (0,1,0) Soil: (50, 40) > (0,0,1) (52, 40) > (0,0,1) … … … (0,0,1) Land Cover Category (Vegetation) 0 (Water) 0 (Soil) A vegetation pixel (10, 89) > (1,0,0)

page 21 of 50 After Training water Vegetation Soil …………………… …………………… x y 1 (Vegetation) 0 (Water) 0 (Soil) A new pixel (x,y), x in band 1, y in band 2 A Well-trained ANN

page 22 of 50 Problems with Traditional Back- Propagation ANN Approaches Time-consuming – Always over iteration, over 5 hours for a small case. Black box - uncontrollable training – Easily trapped in local minimum. Training result unpredictable

page 23 of 50 Optimization Techniques to ANN Approach – Data Representation - Transform to Binary and Gray code format. – Improved Convergence Algorithms - Apply Conjugate Gradient and Resilient Propagation. – Weight initialization - Linear Regression.

page 24 of 50 Data Representation Data representation - Use Binary format and gray code format instead of integer value. Avoid to compute in the saturation range of the activation function Increased the computation space (from 6 to 48) Make the value more smoothly - Gray Code

page 25 of 50 Saturation Range of Activation Function - Sigmoid Function Saturation Original Inputs: 0 ~ 255 New Format: 0 or 1

page 26 of 50 Coding Method Integer format Binary format Forest Pixel 6 Integers48 Integers

page 27 of 50 Gray Code vs. Binary Format Gray Code is another coding system, similar to binary Can better represent continuous value Binary formatGray code formatInteger

page 28 of 50 Improved Convergence algorithms To Make the convergence more robust and reliable, apply: Conjugate Gradient (CG) Resilient Propagation (RPROP)

page 29 of 50 Error Space and Weight Adaptive Total Error Steps Wij Wkl

page 30 of 50 Weight Adaptive with Conjugate Gradient Wij Error Global minimum

page 31 of 50 Resilient Propagation (RPROP)

page 32 of 50 Weight initialization Cut convergence time, do weight initialization using linear regression to pre-process the internal weight, make ANN adapted to the target pattern before training.

page 33 of 50 Data and Experiments Study Area Data Distribution Fine tuning the ANN structure Classification Result

page 34 of 50 Study Area - Rhode Island 1999 Landsat-7 Enhanced Thematic Mapper Plus (ETM+) Image Band 4,3,2 In RGB Band 5,4,3 In RGB

page 35 of 50 Training and Testing Pattern

page 36 of 50 Distribution of Training Data Patterns Plotted by band 3 x 4 Patterns Plotted by band 4 x 5 Deciduous forest Turf / Grass Barren land Conifer forest Forest Wetland Mixed forest Brush Land Non-Forest Wetland Water Urban area

page 37 of 50 ANN Design and Tuning Number of PEs in input layer = Number of spectral Bands in the remote sensing image Number of PEs in output layer = Number of land cover categories Number of PEs in hidden layer(s) to be determined for best performance

page 38 of 50 ANN Design and Tuning - Before And After Optimization

page 39 of 50 Classification Result

page 40 of 50 Classification Result - A Close Look Rhode Island 1999 ETM+ Rhode Island 1999 Land-use and Land-cover map

page 41 of 50 Accuracy Assessment

page 42 of 50 Discussion and Conclusion Accuracy Gray code Vs. Integer format Robust and reliability Discuss on number of hidden layer PEs

page 43 of 50 Accuracy Comparison Non-optimized ANN Accuracy % Optimized ANN Accuracy %

page 44 of 50 Gray-code vs. Integer Value Comparison Between Gray Code and Integer Format

page 45 of 50 Resilient Propagation and Weight Initialization Improvement in robust and reliability

page 46 of 50 Hidden Layer PE Number’s Influence On Classifier’s Performance ( 48 inputs, 150 hidden neurons, 10 output classes) ( 48 inputs, 250 hidden neurons, 10 output classes) ( 48 inputs, 350 hidden neurons, 10 output classes)

page 47 of 50 Hidden Layer PE Number’s Influence On Classifier’s Performance. 350 hidden layer PEs provide the best performance

page 48 of 50 Conclusion Coding method, especially the Gray code is important to Remote Sensing image classification. Optimized ANN provide a robust and reliable classification solution Optimized ANN lead to higher classification accuracy Improved natural resources mapping

page 49 of 50 Acknowledgement The research was funded by NASA (Grant No. NAG5-8829) to Dr. Y.Q. Wang Dr. Y.Q. Wang (major professor) Dr. Pete August (NRS, committee member) Dr. Ken Yang (EE&CE) Dr. Tom Boving (GEO) Lab for Terrestrial Remote Sensing Environmental Data Center

page 50 of 50 Thank You!