D. Shaun Bloomfield 1,2, K. Domijan 3, P. A. Higgins 2, P. T

Slides:

Advertisements

Similar presentations

The blue and green colors are actually the same.

Advertisements

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

H.N. Wang 1 ， H. He 1, X. Huang 1, Z. L. Du 1 L. Y. Zhang 1 and Y. M. Cui 2 L. Y. Zhang 1 and Y. M. Cui 2 1 National Astronomical Observatories 2 National.

Rami Qahwaji & TufanColak EIMC, University of Bradford BD71DP,

Supervised Learning Recap

Minimum Redundancy and Maximum Relevance Feature Selection

1 Operational low visibility statistical prediction Frédéric Atger (Météo-France)

Robust Moving Object Detection & Categorization using self- improving classifiers Omar Javed, Saad Ali & Mubarak Shah.

Lecture 17: Supervised Learning Recap Machine Learning April 6, 2010.

Lecture 14 – Neural Networks

Arizona State University DMML Kernel Methods – Gaussian Processes Presented by Shankar Bhargav.

Classifiers, Part 3 Week 1, Video 5 Classification  There is something you want to predict (“the label”)  The thing you want to predict is categorical.

Shock wave formation heights using 2D density and Alfvén maps of the corona ABSTRACT Coronal shock waves can produce decametric radio emission known Type.

July 11, 2001Daniel Whiteson Support Vector Machines: Get more Higgs out of your data Daniel Whiteson UC Berkeley.

Ch 6. Kernel Methods Pattern Recognition and Machine Learning, C. M. Bishop, Summarized by J. S. Kim Biointelligence Laboratory, Seoul National University.

Logistic Regression L1, L2 Norm Summary and addition to Andrew Ng’s lectures on machine learning.

Hurieh Khalajzadeh Mohammad Mansouri Mohammad Teshnehlab

Matlab Matlab Sigmoid Sigmoid Perceptron Perceptron Linear Linear Training Training Small, Round Blue-Cell Tumor Classification Example Small, Round Blue-Cell.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

Empirical Research Methods in Computer Science Lecture 7 November 30, 2005 Noah Smith.

Logistic Regression Week 3 – Soft Computing By Yosi Kristian.

Spam Detection Ethan Grefe December 13, 2013.

Linear Discrimination Reading: Chapter 2 of textbook.

Evaluating Results of Learning Blaž Zupan

Cross Validation of SVMs for Acoustic Feature Classification using Entire Regularization Path Tianyu Tom Wang T. Hastie et al WS04.

1/18 New Feature Presentation of Transition Probability Matrix for Image Tampering Detection Luyi Chen 1 Shilin Wang 2 Shenghong Li 1 Jianhua Li 1 1 Department.

Feature-based (object-based) Verification Nathan M. Hitchens National Severe Storms Laboratory.

Graham Steward 1, Vasily Lobzin 1, Mike Terkildsen 1, Matt Francis 1, Iver Cairns 2 (1) Bureau of Meteorology, Space Weather Services, Sydney, Australia;

Data Mining and Decision Support

Predicting Flares D. Shaun Bloomfield Astrophysics Research Group Trinity College Dublin Flare Group E, Solar24 9th December 2008 D. Shaun Bloomfield Trinity.

Meeting 8: Features for Object Classification Ullman et al.

Chapter 11 – Neural Nets © Galit Shmueli and Peter Bruce 2010 Data Mining for Business Intelligence Shmueli, Patel & Bruce.

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

D. Shaun Bloomfield 1,2, K. Domijan 3, P. A. Higgins 2, P. T

Sunspot Group Evolution and Flare Forecasting

How to forecast solar flares?

Deep Feedforward Networks

Artificial Neural Networks

An Empirical Comparison of Supervised Learning Algorithms

Machine Learning & Deep Learning

an introduction to: Deep Learning

Instance Based Learning

Marlon Núñez and David A. Núñez Universidad de Málaga

27 April ESANN 2006 Benjamin Schrauwen and Jan Van Campenhout

Table 1. Advantages and Disadvantages of Traditional DM/ML Methods

Evaluating Results of Learning

Restricted Boltzmann Machines for Classification

Announcements HW4 due today (11:59pm) HW5 out today (due 11/17 11:59pm)

An application of machine learning to geomagnetic index prediction: aiding human space weather forecasting Laurence Billingham, Gemma Kelly British Geological.

Understanding the Difficulty of Training Deep Feedforward Neural Networks Qiyue Wang Oct 27, 2017.

Asymmetric Gradient Boosting with Application to Spam Filtering

R-CNN region By Ilia Iofedov 11/11/2018 BGU, DNN course 2016.

Data Mining Classification: Alternative Techniques

Pattern Recognition CS479/679 Pattern Recognition Dr. George Bebis

Pattern Recognition and Image Analysis

Very Deep Convolutional Networks for Large-Scale Image Recognition

Machine Learning 101 Intro to AI, ML, Deep Learning

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

Support Vector Machine I

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

Emerging Active Regions: turbulent state in the photosphere

Machine Learning with Clinical Data

An introduction to: Deep Learning aka or related to Deep Neural Networks Deep Structural Learning Deep Belief Networks etc,

Modeling IDS using hybrid intelligent systems

Introductory Statistics

Machine Learning in Business John C. Hull

Power Regression & Regression estimation of event probabilities (REEP)

Logistic Regression Geoff Hulten.

Presentation transcript:

Solar Flare Forecasting from Magnetic Feature Properties Generated by the SMART Algorithm D. Shaun Bloomfield 1,2, K. Domijan 3, P.A. Higgins 2, P.T. Gallagher 2 1 Northumbria University, UK 2 Trinity College Dublin, Ireland 3 NUI Maynooth, Ireland ESWW13 – 17th Nov 2016

Data Source Solar Monitor Active Region Tracking (SMART) algorithm SOHO/MDI magnetic feature (MF) finder similar in concept to SDO/HMI (S)HARP cut-outs Extracts ~25 magnetic properties for each MF association with GOES >C1.0 NOAA numbered regions and ephemeral/unspotted regions

Training and Testing Data Direct comparison to Ahmed et al. (2013) Sol. Phys., 283, 157 Training (10.75 years): Apr 1996 – Dec 2000 ; Jan 2003 – Dec 2008 Testing (4 years): Jan 2001 – Dec 2002 ; Jan 2009 – Dec 2010 Marginal relevance scores found from training set total length of neutral lines (Lnl) max. horizontal gradient of vertical field across neutral line (Mx_Grad) All-MF NOAA-only Training Testing > C1.0 within 24 hr 16,673 10,571 1,137 707 No-flare or < C1.0 313,617 177,380 5,272 2,789

Classification Rule Construction Random draw of 100 flare (red) and 300 non-flare (black) MFs Build linear logistic regression classifier – sigmoid surface

Classification Rule Variation Repeat previous classifier construction 50 times All-MF case has many zero values (no neutral lines in MF detection) NOAA-only case boundaries much more variable

Forecast Application Categorical forecasts reached by thresholding a classification rule SMART MFs checked against GOES >C1.0 flares within 24 hr Contingency tables drawn up to find categorical skill scores All 50 classification rules applied to every test data point i.e., 50 contingency tables at each classifier threshold value Forecast Flare No-flare Observed TP FN FP TN TSS= TP FN+TP − FP FP+TN HSS= TP+TN − E random N− E random

Forecast Performance All-MFs TSS Classifier Threshold Classif. Thresh. (NOAA) HSS 0.05 0.78 0.29 0.10 0.82 0.37 0.15 0.83 0.42 0.20 0.45 0.25 0.48 0.30 0.81 0.51 0.35 0.80 0.53 0.40 0.55 0.76 0.56 0.50 0.75 0.57 0.73 0.59 0.60 0.70 0.65 0.68 0.61 0.62 0.85 0.90 0.58 0.95 0.44 0.52 All-MFs TSS median 2.5th → 97.5th percentiles Classifier Threshold

Forecast Performance All-MFs TSS NOAA-only TSS Classifier Threshold HSS 0.05 0.78 0.09 0.29 0.13 0.10 0.82 0.50 0.37 0.34 0.15 0.83 0.61 0.42 0.46 0.20 0.65 0.45 0.53 0.25 0.64 0.48 0.56 0.30 0.81 0.62 0.51 0.58 0.35 0.80 0.59 0.40 0.57 0.55 0.76 0.54 0.75 0.73 0.60 0.70 0.68 0.39 0.52 0.36 0.32 0.47 0.85 0.28 0.44 0.90 0.24 0.38 0.95 0.18 0.33 All-MFs TSS median 2.5th → 97.5th percentiles NOAA-only TSS Classifier Threshold

Ordinal log. regression Conclusions Data Forecast method TSS Flare level Reference All-MF Log. regression 0.83 >C1.0 This work Neural network 0.64 Ahmed et al. (2013) NOAA-only 0.65 Ordinal log. regression C-class Song et al. (2009) NOAA/SWPC (human) 0.57 Crown (2012) NOAA/SWPC (look-up) 0.45 McIntosh-Poisson 0.46 Bloomfield et al. (2012) Very good / good performance compared to literature No forecast improvement with more complicated models >2 parameter logistic regression linear classifiers on lower-dimensional projections (e.g., PCA, KPCA) non-linear classifiers (e.g., SVM, GP)