1. ESWW 6 Rami Qahwaji and Tufan Colak School of Computing, Informatics and Media, Bradford University Richmond Road, Bradford BD7 1DP, England, UK. Tel. +44(0)1274 236078 Fax. +44(0)1274 233727 r.s.r.qahwaji@bradford.ac.uk Automated Computer-Based Prediction of Solar Flares: The ASAP System

2. ESWW 6 Introduction ASAP: the principles ASAP: the Implementation ASAP: the Evaluation ASAP: the next-gen Organisation

3. ESWW 6 1. knowledge of the subject domain 2. Accumulated experience over time ( patterns, associations, events ) 3. Ability to generalise when faced by new situations ? Space Weather Predictions Manual predictions are carried out by humans who study recent images and data and take decisions. Humans are aided by:

4. ESWW 6 Benefit of Real time performance More feasible as the size of images and datasets increases (The three solar instruments of SDO, AIA, HMI, and EVE, will produce about 1-2 TB Data/day exceeding the SOHO data volume by more than a thousand time) Consistent objective performance Ability to associate different data sets Ability to extract knowledge from historical data Advantages of Automated Prediction?

5. ESWW 6 ? Challenges for computer-based predictions: ? How to represent human’s knowledge and experiences using computerised rules? How to accurately predict flares if the flaring mechanism is still not totally understood? How to make the computer learn and becomes better with time?

6. ESWW 6 Classical Vs new Growing number of publications using Statistical- based Prediction and solar data mining Physics-based models provide valuable insights into the general flaring mechanisms. But this understanding is still incomplete, which makes it very difficult to develop operational prediction tools. Physics-based Prediction Vs Statistical- based Prediction

7. ESWW 6 On the other hand, statistical models depend on knowledge extraction, large-scale analysis, time series analysis and patterns finding. Hence, they do not require full understanding of the physical processes. But they work more efficiently when loads of data covering many solar cycles (hundreds of years of solar data) exist.

8. ESWW 6 Keeping all this in mind, we were funded by EPSRC to investigate this further. This Research is funded by 2 grants from the Engineering and Physical Sciences Research Council (EPSRC). These grants are: R. Qahwaji, “Image Processing & Machine Learning Techniques for Short-term Prediction of Solar Activity,” EPSRC (GR/T17588/01), 3 years project (10/01/05 till 09/01/08), £124,887. R. Qahwaji, S. Ipson and H. Ugail, “"Image Processing, Machine Learning and Geometric Modelling for the 3D Representation of Solar Features", EPSRC (EP/F022948/1), 3 years project, started 18/02/08, £295,469.

9. ESWW 6 Introduction ASAP: the principles ASAP: the Implementation ASAP: the Evaluation ASAP: the next-gen Organisation

10. ESWW 6 Automated Solar Activity Prediction (ASAP) is a web-compliant, fully automated computer to predict solar flares in near real-time. Several solar imaging algorithms and 5 Neural Network Agents. Fully implemented in C++. Available publically at http://spaceweather.inf.brad.ac.uk What is ASAP?

11. ESWW 6 ASAP is always downloading latest MDI images from SOHO (GIF images) It takes around 10-15 seconds to detect, group and classify sunspots and then to generate flare predictions. It is currently been used to process all the MDI images of SOHO to create the 1 st objective sunspots catalogue for the whole of SOHO’s mission. It is written in C++ and can be modified to run under different platforms (currently runs under Windows and Mac) ASAP Features

12. ESWW 6 The Prediction Model (in a nutshell)

13. ESWW 6 ASAP’s Imaging System SOHO/MDI continuum images are used for the detection of sunspots SOHO/MDI magnetogram images are used for the detection of active regions. [COLAK and QAHWAJI 2008, Sol phy]

14. ESWW 6 ASAP’s Imaging System Stage-1 processing: Applied to both continuum and magnetogram images. Detect the solar disk, determine its radius and centre and remove non relevant information. Calculate the Julian date and solar coordinates (The position angle, heliographic latitude, heliographic longitude). Stage-2 processing: Applied only to magnetogram images. Map the magnetogram image from Heliocentric-Cartesian coordinates to the Carrington Heliographic coordinates. Re-map the image to Heliocentric-Cartesian coordinates. Use centre, radius and solar coordinates of the continuum image as the new centre, radius and solar coordinates of the magnetogram image. [COLAK and QAHWAJI, 2007]

15. ESWW 6 Pre-Processing

16. ESWW 6 ASAP’s Imaging System Detect sunspot candidates from MDI continuum images. Detect active region candidates from MDI magnetogram images. (The MDI magnetogram images show the magnetic fields of the solar photosphere, with black and white areas indicating opposite magnetic polarities.) Apply region growing to combine sunspot and active region candidates. Use neural networks to combine regions of opposite magnetic polarities in order to determine the exact boundaries of sunspot groups. Mark the detected sunspot groups. [COLAK and QAHWAJI 2007]

17. ESWW 6 Sunspot Grouping

18. ESWW 6 ASAP’s Imaging System Extract local features from every sunspot in every group using image processing and neural networks. Extract the length, tallness, and area of the sunspot. Use neural networks to decide the type of penumbra (i.e., Mature or Rudimentary) and whether the sunspot is Symmetric or Asymmetric. Extract features from each sunspot group using image processing. The extracted features are length, largest spot, polarity and distribution. Feed the extracted features to a decision tree to determine the McIntosh classifications.

19. ESWW 6 Sunspot Classification Sunspot groups on the image 04/04/2006 taken at 08:00, the closest classification on the available USAF sunspot catalogue is at 08:32 by San Vito observatory. This observatory detected three sunspot groups and the newly forming group (2) that is detected by our algorithms is not one of them. This group is only available on the sunspot groups detected nearly 9 hours later by Holloman AFB observatory classified as BXO.

20. ESWW 6 The Flares Prediction Model ASAP’s Flares Prediction

21. ESWW 6 * Colak T and Qahwaji R (2009): "ASAP: A Hybrid Computer Platform Using Machine Learning and Solar Imaging for Automated Prediction of Significant Solar Flares" Space Weather, 7, S06001. * Qahwaji R, Colak T, Al-Omari M and Ipson S S(2008): "Automated Machine Learning-Based Prediction of CMEs Based on Flare Associations" Solar Physics, 248 (2): 471-483. * Colak T and Qahwaji R (2008): "Automated McIntosh-based classification of sunspot groups using MDI images" Solar Physics, 248 (2): 277-296. * Qahwaji R and Colak T (2007): "Automatic Short-Term Solar Flare Prediction Using Machine Learning And Sunspot Associations" Solar Physics, 241 (1): 195-211. Publications related to ASAP’s developments stages

22. ESWW 6 Introduction ASAP: the principles ASAP: the Implementation (transforming ASAP from a scientific model to an operational system) ASAP: the Evaluation ASAP: the next-gen Organisation

23. ESWW 6 http://spaceweather.inf.brad.ac.uk/ Automated predictions of solar flares are provided daily in our website

24. ESWW 6 ASAP: Automated Solar Activity Prediction The flares prediction window of ASAP. ASAP is Available freely for download at our website

25. ESWW 6 ASAP is integrated with SWENET

26. ESWW 6 ASAP is also integrated with the Integrated Space Weather Analysis System of NASA

27. ESWW 6

28. Introduction ASAP: the principles ASAP: the Implementation ASAP: the Evaluation ASAP: the next-gen Organisation

29. ESWW 6 Colak T and Qahwaji R (2009), Space Weather The system was tested on MDI images from February 1, 1999 to December 31, 2002. Trained on associations with 24 hours difference. QR is used to calculate the accuracy in probability predictions. POD measures the probability of actual solar flares being predicted correctly by the hybrid system. FAR measures the probability of the hybrid system predicting a solar flare that actually does not occur. PC measures the correct prediction rate of the overall system. HSS is a measure showing the chance factor in predictions. HSS can range from -1 (for no correct predictions) to +1 (for all correct predictions) and a value of zero indicates that the predictions have been generated mainly by chance.

30. ESWW 6 How accurate our predictions? (Confidence Level?) The accuracy of our predictions depends mostly on accuracy of each stage. We have an accuracy of ~95% on sunspot grouping, ~80% on sunspot group classification and ~90% on flare prediction depending on correct classification. In total this means we have a success rate of ~70% on final flare prediction when we combine all stages.

31. ESWW 6 Introduction ASAP: the principles ASAP: the Implementation ASAP: the Evaluation ASAP: the next-gen Organisation

32. ESWW 6 The next gen should provide REAL physical insight into flares eruptions. Active regions will be Characterized by their magnetic properties (i.e. Magnetic flux, Flux, imbalances, energies, Neutral line properties, etc.) Hence combining physical-based modelling with statistical-based modelling

33. ESWW 6 ASAP needs Improvements! Errors in the catalogues can effect the predictions. ASAP does not provide real physical insight on the causes of flares eruptions Modifications to work with SDO

34. ESWW 6 On going Work! New methods for modelling the energy of active regions. Studying the evolution of sunspots using HMM.

35. ESWW 6 Thank You