1. Canadian Weather Analysis Using Connectionist Learning Paradigms Imran Maqsood*, Muhammad Riaz Khan , Ajith Abraham  * Environmental Systems Engineering Program, Faculty of Engineering, University of Regina, Regina, Saskatchewan S4S 0A2, Canada, E-mail: maqsoodi@uregina.ca  Partner Technologies Incorporated, 1155 Park Street, Regina, Saskatchewan S4N 4Y8, Canada, E-mail:riaz917@hotmail.com  Faculty of Information Technology, School of Business Systems, Monash University, Clayton 3800, Australia, E-mail: ajith.abraham@ieee.org 7th Online World Conference on Soft Computing in Industrial Applications (on WWW), September 23 - October 4, 2002

2. 23 Sep - 04 Oct, 2002WSC72 CONTENTS Introduction MLP, ERNN and RBFN Background Experimental Setup of a Case Study Test Results Conclusions

3. 23 Sep - 04 Oct, 2002WSC73 1. INTRODUCTION Weather forecasts provide critical information about future weather Weather forecasting remains a complex business, due to its chaotic and unpredictable nature Combined with threat by the global warming and green house gas effect, impact of extreme weather phenomena on society is growing costly, causing infrastructure damage, injury and the loss of life.

4. 23 Sep - 04 Oct, 2002WSC74 Accurate weather forecast models are important to the countries, where the entire agriculture depends upon weather Previously, several artificial intelligence techniques have been used in the past for modeling chaotic behavior of weather However, several of them use simple feed-forward neural network training methods using backpropagation algorithm

5. 23 Sep - 04 Oct, 2002WSC75 Study Objectives To develop an accurate and reliable predictive models for forecasting the weather of Vancouver, BC, Canada. To compare performance of multi-layered perception (MLP) neural networks, Elman recurrent neural networks (ERNN) and radial basis function network (RBFN) for the weather analysis.

6. 23 Sep - 04 Oct, 2002WSC76 2. ANN BACKGROUND INFORMATION ANN Advantages An ability to solve complex and non-linear problems Quick response Self-organization Real time operation Fault tolerance via redundant information coding Adaptability and generalization

7. 23 Sep - 04 Oct, 2002WSC77 O1O1 O2O2 OmOm VjVj W ij w jk    Input Layer Hidden Layer Output Layer I1I1 I2I2 InIn kk OiOi (a) Multi-Layered Perceptron (MLP) Networks network is arranged in layers of neurons every neuron in a layer computes sum of its inputs and passes this sum through a nonlinear function as its output. Each neuron has only one output, but this output is multiplied by a weighting factor if it is to be used as an input to another neuron (in a next higher layer) There are no connections among neurons in the same layer. Figure: Architecture of 3-layered MLP network for weather forecasting

8. 23 Sep - 04 Oct, 2002WSC78 (b) Elman Recurrent Neural Networks (ERNN) ERNN are a subclass of recurrent networks They are multilayer perceptron networks augmented with one or more additional context layers storing output values of one of the layers delayed by one step and used for activating this or some other layer in the next time step The Elman network can learn sequences that cannot be learned with other recurrent neural network    I3I3   Input LayerHidden LayerOutput Layer I1I1 I2I2 InIn O1O1 OmOm D -1 Feedback Figure: Architecture of 3-layered ERNN

9. 23 Sep - 04 Oct, 2002WSC79 (c) Radial Basis Function Network (RBFN) network consists of 3-layers: input layer, hidden layer, and output layer The neurons in hidden layer are of local response to its input and known as RBF neurons, while the neurons of the output layer only sum their inputs and are called linear neurons network is inherently well suited for weather prediction, because it naturally uses unsupervised learning to cluster the input data. W2W2 WnWn W0W0 W1W1     11 22 nn I1I1 I2I2 ImIm Input Layer Pure linear Output =  w i  i (x) Adjustable weights w i w 0 = bias Adjustable centers c i Adjustable spreads  i Figure: Architecture of RBFN

10. 23 Sep - 04 Oct, 2002WSC710 3. EXPERIMENTAL SETUP Weather Data: Vancouver, BC, Canada 1-yr data: Sep 2000 – Aug 2001 Observed Parameters (most important): –Minimum Temperature ( o C) –Maximum Temperature( o C) –Wind-Speed (km/hr)

11. 23 Sep - 04 Oct, 2002WSC711 Training and Testing Datasets Dataset 1: MLP and ERNN –Testing dataset = 11-20 January 2001 –Training dataset = remaining data Dataset 2: RBFN, MLP and ERNN –Testing dataset = 01-15 April 2001 –Training dataset = remaining data We used this above method to ensure that there is no bias on the training and test datasets

12. 23 Sep - 04 Oct, 2002WSC712 Simulation System Used Pentium-III, 1GHz processor 256 MB RAM all the experiments were simulated using MATLAB Steps taken before starting the training process: Error level was set to a value (10 -4 ) The hidden neurons were varied (10-80) and the optimal number for each network were then decided.

13. 23 Sep - 04 Oct, 2002WSC713 Convergence of the LM and OSS training algorithms using MLP network Convergence of the LM and OSS training algorithms using ERNN 4. TEST RESULTS Training Convergence of MLP and ERNN

14. 23 Sep - 04 Oct, 2002WSC714 Comparison of Actual vs. 10-day ahead Forecasting using OSS and LM approaches MLP network ERNN (a) Minimum Temperature (11-20 Jan 2001) Performance evaluation parameters (min. temperature) MLP Network ERNN OSSLM OSSLM Mean absolute % error (MAPE)0.02210.0202 0.01820.0030 Root mean square error (RMSE)0.0199 0.0031 Mean absolute deviation (MAD)0.76510.8411 0.72310.1213 Correlation coefficient0.96570.9940 0.98260.9998 Training time (minutes)0.31 7 Number of iterations (epochs)10157 67311

15. 23 Sep - 04 Oct, 2002WSC715 MLP network ERNN (b) Maximum Temperature (11-20 Jan 2001) Performance evaluation parameters MLP Network ERNN OSSLM OSSLM Mean absolute % error (MAPE)0.01700.0087 0.01650.0048 Root mean square error (RMSE)0.02000.0099 0.01990.0067 Mean absolute deviation (MAD)0.81750.4217 0.79440.2445 Correlation coefficient0.9640.999 0.9450.982 Training time (minutes)0.430 1.830 Number of iterations (epochs)8507 113510

16. 23 Sep - 04 Oct, 2002WSC716 MLP network ERNN (c) Maximum Wind-Speed (11-20 Jan 2001) Performance evaluation parameters (wind-speed) MLP Network ERNN OSSLM OSSLM Mean absolute % error (MAPE)0.08960.0770 0.08730.0333 Root mean square error (RMSE)0.19890.0162 0.01990.0074 Mean absolute deviation (MAD)0.82970.6754 0.76180.3126 Correlation coefficient0.97140.9974 0.98860.9995 Training time (minutes)0.31 0.58 Number of iterations (epochs)8518 120812

17. 23 Sep - 04 Oct, 2002WSC717 Comparison of Relative Percentage Error between Actual and Forecasted Parameters MLP network ERNN Minimum Temperature Maximum Temperature Wind-Speed

18. 23 Sep - 04 Oct, 2002WSC718 Comparison of Training of Connectionist Models Network modelNumber of hidden neurons Number of hidden layers Activation function used in hidden layer Activation function used in output layer MLP 451Log-sigmoidPure linear ERNN451Tan-sigmoidPure linear RBFN1802Gaussian functionPure linear

19. 23 Sep - 04 Oct, 2002WSC719 0 5 10 15 03691215 Days of the Month Temperature ( o C) 0 5 10 15 03691215 Days of the Month Temperature ( o C) Actual valueRBFNMLPRNN Maximum Temperature Minimum Temperature Wind-Speed Comparison among three Neural Networks Techniques for 15-day ahead Forecasting (1-15 Apr 2001)

20. 23 Sep - 04 Oct, 2002WSC720 Model Performance Evaluation Parameters Maximum Temperature Minimum Temperature Wind Speed RBFN MAP MAD Correlation Coefficient 3.821 0.420 0.987 3.622 1.220 0.947 4.135 0.880 0.978 MLP MAP MAD Correlation Coefficient 6.782 1.851 0.943 6.048 1.898 0.978 6.298 1.291 0.972 ERNN MAP MAD Correlation Coefficient 5.802 0.920 0.946 5.518 0.464 0.965 5.658 0.613 0.979 Performance Evaluation of RBFN, MLP and ERNN Techniques

21. 23 Sep - 04 Oct, 2002WSC721 5. CONCLUSIONS In this paper, we developed and compared the performance of multi-layered perceptron (MLP) neural network, Elman recurrent neural network (ERNN) and radial basis functions network (RBFN). It can be inferred that ERNN could yield more accurate results, if good data selection strategies, training paradigms, and network input and output representations are determined properly.

22. 23 Sep - 04 Oct, 2002WSC722 Levenberg-Marquardt (LM) approach appears to be the best learning algorithm. However, it requires more memory and is computationally complex while compared to one-step-secant (OSS) algorithm. Empirical results clearly demonstrate that compared to MLP neural network and ERNN, RBFN are much faster and more reliable for the weather forecasting problem considered. A comparison of the neurocomputing techniques with other statistical techniques would be another future research topic.

23. 23 Sep - 04 Oct, 2002WSC723 THANK YOU !