2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 1 COST 724 WHY NEURAL NETWORKS? E. Tulunay 1, Y. Tulunay 2 ODTU / METU 1.Dept. of Electrical and Electronics Eng. 2.Dept. of Aerospace Eng , Ankara, TURKEY

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 2 OUTLINE –Neural Networks –METU-NN Training, validation in training, validation in test –NN Applications –Space Weather-borne processes –A case study example: Forecasting Total Electron Content Maps by Neural Network Technique –Conclusions –References

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 3 Neural Networks (NN) NN: system of interconnected computational elements operating in parallel, arranged in patterns similar to biological NNs and modeled after the human brain [Rumelhart et al., 1986]. Since 1990’s, interest in NNs has increased mainly because of the developments in very large scale integrated circuit technology, optical devices and new learning paradigms which make rapid and inexpensive implementation possible.

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 4 Neuron : information processing unit consisting of connecting links, adder and activation function. Adder : for summing bias and the input signals weighted in neuron’s connecting links. It follows an activation function for limiting the amplitude of the neuron’s output [Haykin, 1999]. ANN is a system of inter-connected computational elements, the neurons, operating in parallel, arranged in patterns similar to biological neural networks and modeled after the human brain [Tulunay, 1991]. [Y. Tulunay et al., 2004-a]

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 5 Biological to Artificial Neurons Tulunay, 2004 METU-NN The multilayer perceptron - feed forward NNs - neurons arranged in layers.

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 6 Training, validation in training, validation in test In training, weights - initially set to arbitrary values. The inputs - applied and then NN produces an output. The difference between the NN output and the desired output : the error. During training, weights - adapted to minimize the error by using various algorithms. Memorization is avoided. After the original software was developed in order to implement this algorithm, the training process - completed and these trained NNs - ready to perform the forecasting function [Altinay et al., 1997].

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 7 NN applications in various fields including: adaptive pattern recognition, adaptive signal processing, adaptive dynamic modeling, adaptive control, optimization, expert systems and Earth System Science applications. Some other specific applications include control of robot arm, diagnosis and numeric to symbolic conversion [Tulunay, 1991].

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 8 NN based models are generic in that they have been applied to variety of several processes. The only requirement for NN application is the availability of representative data Relative Significances of Inputs

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 9 Unpredictable variability of the ionospheric parameters due to Space Weather-borne processes significant effects on both space and Earth based systems such as communication, radar, navigation etc. Space Weather has significant effects on Earth climate, weather and on biological systems including human health. Therefore, forecasting Near-Earth Space parameters, especially during the disturbed Space Weather conditions, is crucial.

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 10 It is most desirable to drive mathematical forecasting and mapping models based on physics. However, this is very complex and prohibitively difficult task since Space Weather processes are non-linear and time-varying. It has been demonstrated that the data driven approaches such as the use of NN based methods are promising in modeling of ionospheric processes.

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 11 In particular, the NN models are promising in forecasting applications under disturbed conditions, eg. [Williscroft and Poole., 1996; Tulunay et al., 1997; Altinay et al., 1997; Cander et al., 1998; Wintoft and Cander, 1999; Francis et al, 2000; Y. Tulunay et al., 2001; Vernon and Cander, 2002; E. Tulunay et al., 2004-a; Y. Tulunay et al., 2004-a; Y. Tulunay et al., 2004-b; Radicella and Tulunay, 2004; Stamper et al., 2004; McKinnell and Poole, 2004; E. Tulunay et al., 2006-a]. Space weather centers provide forecasts of solar and geophysical parameters. As an example, the Lund Space Weather Center uses artificial intelligence (AI) to forecast Kp parameters [Boberg et al., 2000].

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 12 Since the 1990’s a small group at the METU in Ankara has been working on data-driven generic models of Earth System processes. The METU-NN model has been applied to Near- Earth Space processes for a variety of tasks including the forecasting and mapping of the foF2 and TEC [Tulunay et al., 2000; Tulunay et al., 2001, E. Tulunay et al., 2006-a]. Some recent applications include forecasting of solar flux bursts [Y. Tulunay et al., 2005-a] and Schumann resonances [E. Tulunay et al., 2006-b].

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 13 METU Models on NES processes: 1. Temporal and Spatial Forecasting of the ionospheric critical frequencies, foF2 [Tulunay et al., 2000] 2. Forecasting Total Electron Content Maps by Neural Network Technique [Tulunay et al., 2006-a; Ciraolo, 2004] 3. Forecasting GPS TEC Using the Neural Network Technique “A Further Demonstration” [E. Tulunay et al., 2004] 4. An Attempt to Model the Influence of the Trough on HF Communication by Using Neural Network [Tulunay et al., 2001] 5. Forecasting Magnetopause Crossing Locations by Using Neural Networks [Tulunay et al., 2005-b] 6. The ELF Characterization of the Earth-Ionosphere Cavity: Forecasting the Schumann Resonances (SR) [Tulunay et al., 2006-b] 7. Timed EUV Flux Data and the METU-NN Model 8. Neural Network Modeling in Forecasting the Near Earth Space Parameters: Forecasting of the Solar Radio Fluxes [Tulunay et al., 2005-a]

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 14 A Case Study Example: Forecasting Total Electron Content Maps by Neural Network Technique [Tulunay et al., 2006-a; Ciraolo, 2004] In order to understand more about the complex response of the magnetosphere and ionosphere to extreme solar events, the METU-NN model was used in connection with the series of space weather events in November TEC values of the ionosphere were forecast during these space weather events. In order to facilitate an easier interpretation of the forecast TEC values, maps of TEC are produced by using the Bezier surface-fitting technique

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 15 Observed GPS TEC data [Ciraolo, 2004] : - Ten minute vertical TEC data between 1 Nov and 11 Dec grid locations centered over Italy between latitudes of (35.5º N; 47.5º N) and longitudes of (5.5º E; 19.5º E)

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 16 METU-NN technique - to forecast TEC grid values - surfaces in mapping the forecast grid values

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 17 -Figure 1. Observed (dotted) and 1 hour ahead forecast (solid) TEC during 16 Nov :10 UT - 29 Nov :00 UT for single grid point (13.5˚ E; 41.5˚ N) Results

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 18 - Figure 2. Scatter Diagram (dots) with best-fit line (solid) for the 1 hour ahead Forecast mapping and Observed TEC values for single grid point (13.5˚ E; 41.5˚ N) for 20 November 2003

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 19 METU-NN technique - to forecast TEC grid values - surfaces in mapping the forecast grid values

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 20 -Video 1. Observed and 1 hour ahead forecast TEC Maps by METU-NN during the afternoon of 20 Nov. 2003

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 21 - Figure 3. Absolute Error Map of observed and 1 hour ahead forecast TEC during Nov. 2003

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 22 Table 1. Error Table for 1 h in advance forecasts by METU-NN for the validation time period (16-29 Nov. 2003) 1 TECu = el./m 2 The forecast mapping error values - within operational tolerance [Radicella, 2004]. Location 11.5˚E 38.5˚N 13.5˚E 41.5˚N 15.5˚E 44.5˚N Overall TEC Map Absolute Error (TECu) Normalized Error (%) Cross Correlation Coeff. (x10 -2 )

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 23 Conclusions -The system reached correct operating point during training -METU-NN learned the inherent nonlinearities of the process -Representable data in inputs + Proper construction of METUNN = complex nonlinear processes are modeled -LAN access for the METU models on NES processes except TEC Mapping are available. Web access will also be available when COST 724 web is operative [Ozkok (supervisor: E. Tulunay; co-supervisor: Y. Tulunay), 2005].

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 24 References Altinay O., E.Tulunay, and Y. Tulunay (1997), Forecasting of ionospheric critical frequency using neural networks, Geophysical Research Letter, 24(12), , and COST251 TD(96)016. Boberg F., P. Wintoft, and H. Lundstedt (2000), Real time Kp predictions from solar wind data using neural networks, Phys. Chem. Earth, Pt C, 25(4), Cander Lj.R., M.M. Milosavlijevic, S.S. Stankovic, and S. Tomasevic (1998), Ionospheric Forecasting Technique by Artificial Neural Network, Electronics Letters, 34(16), Online No: , Ciraolo G. (2004), Private communication. T. Dudok de Wit, J. Lilensten, J. Aboudarham, P.-O. Amblard, and M. Kretzschmar (2005), Retrieving the solar EUV spectrum from a reduced set of spectral lines, Annales Geophysicae, 23, 3055–3069. Francis, N.M., P.S. Cannon, A.G. Brown, and D.S. Broomhead (2000), Nonlinear prediction of the ionospheric parameter foF2 on hourly, daily, and monthly timescales, J. Geophys. Res., 105(A6), Hagan M.T., and M.B. Menhaj (1994), Training Feedforward Networks with the Marquard Algorithm, IEEE Transactions on Neural Networks, 5(6), Haykin S. (1999), Neural Networks: A Comprehensive Foundation, 2nd ed., pp. 2, 10, 21-22, 83-84, 169, 215, Prentice-Hall, Inc., New Jersey, USA. Jackson, J.H. (1958), On localization, in Selected writings, 2, Basic Books, New York, Original work Kalnay et al. (1996), The NCEP/NCAR 40-year reanalysis project, Bull. Amer. Meteor. Soc., Kaya, A. (1998), The effects of interplanetary magnetic field, IMF on the Earth space radio systems M.S. Thesis, supervisor: Y. Tulunay, Aeronaut. Eng. Dep., Middle East Tech. Univ., Ankara, Turkey, Dec., LASCO web site, (2002), Large Angle and Spectrometric Coronagraph Experiment (LASCO) and Solar and Heliospheric Observatory (SOHO (ESA & NASA)) Extreme ultraviolet Imaging Telescope (EIT) instruments, Coronal Mass Ejections, the SOHO/LASCO data produced by a consortium of the Naval Research Laboratory (USA), Max-Planck-Institut fuer Aeronomie (Germany), Laboratoire d'Astronomie (France), and the University of Birmingham (UK)., Luira A.R. (1966), Higher cortical functions in man, Basic Books, New York. McKinnell L.A., and A.W.V. Poole (2004), Predicting the ionospheric F layer using neural networks, J. Geophys. Res., 109(A8), A08308, doi: /2004JA Ozkok Y. (2005), Web Based Ionospheric Forecasting Using Neural Network and Neurofuzzy Models, (Supervisor: E. Tulunay, Co-supervisor: Y. Tulunay), MS Thesis, EEE Dept., Middle East Technical University, Ankara, Turkey, July 2005

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 25 Radicella S.M., and E. Tulunay (2004), Space plasma effects on Earth-space and satellite-to-satellite communications: Working Group 4 overview, Annals of Geophysics, 47(2/3), Rumelhart D.E., J.L. McClelland and the PDP Research Group (1986), Parallel distributed processing, explorations in the microstructure of cognition, Vol. 1: Foundations, A Bradford Book, The MIT Press, Cambridge, Mass, and London, 1986, Stamper R., A. Belehaki, D. Buresova, Lj.R. Cander, I. Kutiev, M. Pietrella, I. Stanislawska, S. Stankov, I. Tsagouri, Y.K. Tulunay, and B. Zolesi (2004), Nowcasting, forecasting and warning for ionospheric propagation: tools and methods, Annals of Geophysics, 47(2/3), Tulunay E. (1991), Introduction to Neural Networks and their Application to Process Control, in Neural Networks Advances and Applications, edited by E. Gelenbe, , Elsevier Science Publishers B.V., North-Holland. Tulunay E., C. Ozkaptan, Y. Tulunay (2000), Temporal and Spatial Forecasting of the foF2 Values up to Twenty four Hour in Advance, Phys. Chem. Earth, 25(4), Tulunay E., E.T. Senalp, Lj.R. Cander, Y.K. Tulunay, A.H. Bilge, E. Mizrahi, S.S. Kouris, N. Jakowski (2004-a), Development of algorithms and software for forecasting, nowcasting and variability of TEC, Annals of Geophysics, 47(2/3), Tulunay E., Y. Tulunay, E.T. Senalp, Lj.R. Cander (2004-b), Forecasting GPS TEC Using the Neural Network Technique “A Further Demonstration”, Bulgarian Geophysical Journal, 30(1-4), Tulunay E., E.T.Senalp, S.M.Radicella, Y.Tulunay (2006-a), Forecasting Total Electron Content Maps by Neural Network Technique, Radio Science, Vol. 41, No. 4, RS4016, American Geophysical Union, Washington, USA. Tulunay E., Y. Tulunay, E. Altuntas, E.T Senalp, Y. Bahadirlar (2006-b), Neural Network Forecasting Of Schumann Resonances In The Near Earth Space, Third European Space Weather Week (ESWW3), November 2006, Royal Library of Belgium, Brussels, Belgium. Tulunay, Y., and J.M. Grebowsky (1978), The noon and midnight mid-latitude trough as seen by Ariel 4, J. Atmos. Terr.Phys., 40, Tulunay, Y., A. Kaya, and G. Oke (1997), Further possible effect of the IMF turnings on the Slough critical frequencies and the signature of the electron density trough on the COST251 area, paper presented at Fifth Management Committee Meeting of COST251 and Joint COST/IRI Workshop, Inst. of Atmos. Phys., Kuehlungsborn, Germany, May Tulunay Y., E. Tulunay, and E.T. Senalp (2001), An Attempt to Model the Influence of the Trough on HF Communication by Using Neural Network, Radio Science, 36(5), , September - October Tulunay Y., E. Tulunay, and E.T. Senalp (2004-a), The Neural Network Technique-1: A General Exposition, Adv. Space Res., 33(6),

2007NEURAL NETWORKS, E. Tulunay, Y. Tulunay 26 Tulunay Y., E. Tulunay, and E.T. Senalp (2004-b), The Neural Network Technique-2: An Ionospheric Example Illustrating its Application, Adv. Space Res., 33(6), Tulunay Y., M. Messerotti, E.T. Senalp, E. Tulunay, M. Molinaro, Y.I. Ozkok, T. Yapici, E. Altuntas, N. Cavus (2005-a), Neural Network Modeling in Forecasting the Near Earth Space Parameters: Forecasting of the Solar Radio Fluxes, COST 724: "Developing the scientific basis for monitoring, modeling and predicting Space Weather" Scientific Workshop, Proceedings CD, October 2005, Athens, Greece. Tulunay Y., D.G. Sibeck, E.T. Senalp, E. Tulunay (2005-b), Forecasting magnetopause crossing locations by using Neural Networks, Adv. Space Res., 36(12) Vernon A., and Lj.R. Cander (2002), Regional GPS receiver networks for monitoring local mid-latitude total electron content, Annals of Geophysics, 45(5), Wang, J.S., Chen, Y.P. (2006), A Fully Automated Recurrent Neural Network for Unknown Dynamic System Identification and Control, IEEE T. Circuits Syst.- I: Regular Papers, 53-6, pp Williscroft L.A., and A.W.V. Poole (1996), Neural Networks, foF2, sunspot number and magnetic activity, Geophys. Res. Lett., 23(24), Wintoft P, Lj.R. Cander, (1999). Short term prediction of foF2 using timedelay neural networks, Phys. Chem. Earth, Pt C, 24(4),