Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035.

of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035 tren8835@postoffice.uri.edu 10/07/2002

of 50 Outline Introduction of Satellite Remote Sensing Remote Sensing Image Classification Methodology Data and Experiments Discussion and Conclusion

of 50 Earth Surface and Land Cover Map How to achieve an accurate land cover map in natural resources research?

of 50 Remote Sensing Remote Sensing is defined as: The science of acquiring, processing and interpreting images that record the interaction between electromagnetic energy and matter.

of 50 Different Type of Land Use and Land cover and Spectral Characteristics Blue Band Red BandNear-infrared Band Green Band

of 50 From Digital Image to Land-use Map Visualization Classification Multi-spectral Digital Image Pseudo Color Image Land-use and Land-cover Map

of 50 Classification System In this study, we used the USGS classification with 10 land cover categories: 1. Turf/Grass 2. Barren land 3. Conifer forest 4. Deciduous forest 5. Mixed forest 6. Brush land 7. Urban land 8. Water 9. Non-forest wetland 10. Forest wetland

of 50 Remote Sensing Image Classification Statistical Methods – Supervised – Unsupervised Artificial Neural Network Approaches Rule-based...

of 50 Supervised Remote Sensing Image Classification Vegetation Water Soil Band 1 (0 ~ 255) Band 2 (0 ~ 255)

of 50 Classification Process Mapping Relationship Methods: Statistical classifier ANN-based classifier Observation space Solution space Landsat TM Band1 Band2 Band3 Band4 Band5 Band6 Band7 0~255 0~255 0~255 0~255 0~255 0~255 0~255 Category 1 Category.. Category … Category N Water wetland Forest Agri. Urban Residential 40 45 61 193 80 112 25 Category: Forest (Pattern)

of 50 Statistical Methods - Need Gaussian (Normal) distribution on the input data which is required by Bayesian classifier. - Restrictions about the format of input data. Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Water Soil

of 50 Statistical Methods: Deciduous forest Conifer forest Forest Wetland Mixed forest Brush Land Non-forest Wetland How to find a boundary for the following patterns?

of 50 Artificial Neural Network Approach No need for normal distribution on input data Flexibility on input data format Improved classification accuracy Robust and reliability

of 50 ………………………… …………………… Artificial Neural Network Is Defined by... Processing elements Organized topological structure Learning rules

of 50 Processing Element (PE)  f(x) Output Input PE Artificial counterparts of neurons in a brain W j1 W j2 W j4 W j3 W j5

of 50 PE’s Output Function of Processing Elements ………………………… …………………… unit j o1o1 o2o2 o3o3 f w j1 w j2 w j3 ojoj PE’s Inputs Receive outputs from each PEs locate in previous layer. Compute the output with a Sigmoid activation function F( Sumof( O i * W ji) ) Transfer the output to all the PEs in next layer

of 50 Input layerHidden layerOutput layer ………………………… …………………… Input vector i(x 1, x 2, … x n ) Output vector i(o 1, o 2, … o m ) Artificial Neural Network Architecture - Backpropagation ANN (BPANN) Feed InformationAnalysis Result

of 50 Pattern Recognition Vegetation: (10, 89) ------> (1,0,0) (11,70) ------> (1,0,0) … … … (1,0,0) Water: (10, 21) ------> (0,1,0) (15, 32) ------> (0,1,0) … … … (0,1,0) Soil: (50, 40) -------> (0,0,1) (52, 40) -------> (0,0,1) … … … (0,0,1) Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Wa t er Soil

of 50 ANN Design Pattern Input layerHidden layerOutput layer …………………… How many PEs we need - Basic rules in designing an ANN. Input layer PEs - by dimension of input vector Output layer PEs - by total number of patterns (classes) …………………… Feed Forward Back-Propagate (0 ~ 255) (0 ~ 1) Band 1 (0 ~ 255) Band 2 (0 ~ 255) Vegetation Water Soil

of 50 ANN Training - From Pattern to Land Cover Category Pattern …………………… …………………… Feed Forward Back-Propagate Vegetation: (10, 89) ------> (1,0,0) (11,70) ------> (1,0,0) … … … (1,0,0) Water: (10, 21) ------> (0,1,0) (15, 32) ------> (0,1,0) … … … (0,1,0) Soil: (50, 40) -------> (0,0,1) (52, 40) -------> (0,0,1) … … … (0,0,1) Land Cover Category 10 89 1 (Vegetation) 0 (Water) 0 (Soil) A vegetation pixel (10, 89) ------> (1,0,0)

of 50 After Training water Vegetation Soil …………………… …………………… x y 1 (Vegetation) 0 (Water) 0 (Soil) A new pixel (x,y), x in band 1, y in band 2 A Well-trained ANN

of 50 Problems with Traditional Back- Propagation ANN Approaches Time-consuming – Always over 10000 iteration, over 5 hours for a small case. Black box - uncontrollable training – Easily trapped in local minimum. Training result unpredictable

of 50 Optimization Techniques to ANN Approach – Data Representation - Transform to Binary and Gray code format. – Improved Convergence Algorithms - Apply Conjugate Gradient and Resilient Propagation. – Weight initialization - Linear Regression.

of 50 Data Representation Data representation - Use Binary format and gray code format instead of integer value. Avoid to compute in the saturation range of the activation function Increased the computation space (from 6 to 48) Make the value more smoothly - Gray Code

of 50 Saturation Range of Activation Function - Sigmoid Function Saturation Original Inputs: 0 ~ 255 New Format: 0 or 1

of 50 Coding Method 40 45 61 193 80 112 25 Integer format Binary format 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 1 1 0 0 1 Forest Pixel 6 Integers48 Integers

of 50 Gray Code vs. Binary Format Gray Code is another coding system, similar to binary Can better represent continuous value Binary formatGray code formatInteger 127 0 1 1 1 1 1 1 1 0 1 0 0 1 0 0 0 128 1 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0

of 50 Improved Convergence algorithms To Make the convergence more robust and reliable, apply: Conjugate Gradient (CG) Resilient Propagation (RPROP)

of 50 Error Space and Weight Adaptive Total Error Steps Wij Wkl

of 50 Weight Adaptive with Conjugate Gradient Wij Error Global minimum

of 50 Resilient Propagation (RPROP)

of 50 Weight initialization Cut convergence time, do weight initialization using linear regression to pre-process the internal weight, make ANN adapted to the target pattern before training.

of 50 Data and Experiments Study Area Data Distribution Fine tuning the ANN structure Classification Result

of 50 Study Area - Rhode Island 1999 Landsat-7 Enhanced Thematic Mapper Plus (ETM+) Image Band 4,3,2 In RGB Band 5,4,3 In RGB

of 50 Training and Testing Pattern

of 50 Distribution of Training Data Patterns Plotted by band 3 x 4 Patterns Plotted by band 4 x 5 Deciduous forest Turf / Grass Barren land Conifer forest Forest Wetland Mixed forest Brush Land Non-Forest Wetland Water Urban area

of 50 ANN Design and Tuning Number of PEs in input layer = Number of spectral Bands in the remote sensing image Number of PEs in output layer = Number of land cover categories Number of PEs in hidden layer(s) to be determined for best performance

of 50 ANN Design and Tuning - Before And After Optimization

of 50 Classification Result

of 50 Classification Result - A Close Look Rhode Island 1999 ETM+ Rhode Island 1999 Land-use and Land-cover map

of 50 Accuracy Assessment

of 50 Discussion and Conclusion Accuracy Gray code Vs. Integer format Robust and reliability Discuss on number of hidden layer PEs

of 50 Accuracy Comparison Non-optimized ANN Accuracy - 79.58% Optimized ANN Accuracy - 92.27%

of 50 Gray-code vs. Integer Value Comparison Between Gray Code and Integer Format

of 50 Resilient Propagation and Weight Initialization Improvement in robust and reliability

of 50 Hidden Layer PE Number’s Influence On Classifier’s Performance. 48-150-10 ( 48 inputs, 150 hidden neurons, 10 output classes) 48-250-10 ( 48 inputs, 250 hidden neurons, 10 output classes) 48-350-10 ( 48 inputs, 350 hidden neurons, 10 output classes)

of 50 Hidden Layer PE Number’s Influence On Classifier’s Performance. 350 hidden layer PEs provide the best performance

of 50 Conclusion Coding method, especially the Gray code is important to Remote Sensing image classification. Optimized ANN provide a robust and reliable classification solution Optimized ANN lead to higher classification accuracy Improved natural resources mapping

of 50 Acknowledgement The research was funded by NASA (Grant No. NAG5-8829) to Dr. Y.Q. Wang Dr. Y.Q. Wang (major professor) Dr. Pete August (NRS, committee member) Dr. Ken Yang (EE&CE) Dr. Tom Boving (GEO) Lab for Terrestrial Remote Sensing Environmental Data Center

of 50 Thank You!

Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035.

Similar presentations

Presentation on theme: "Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035.

Similar presentations

Presentation on theme: "Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401) 874-9035."— Presentation transcript:

Similar presentations

About project

Feedback