1. Flare scoreboard planning group: J. Andries (SIDC), A. Devos (SIDC), J. Guerra (TCD), S. Hong (KSWC), M. Kuznetsova (NASA GSFC), L. Mays (NASA GSFC), S. Murray (Met Office), G. Steward (BoM), M. Terkildsen (BoM). Real time forecasting methods validation with the Flare Scoreboard The Flare Scoreboard facilitates model validation by collecting and displaying real-time forecasts from multiple models and serving data downloads via an API. Currently registered models: ASSA Automatic Solar Synoptic Analyser ASAP Automatic Solar Activity Prediction BoM Data-driven probabilistic flare forecast model MAG4 LOS and vector magnetogram forecasts Met Office Radio Blackout Forecast SIDC Human operator moderated forecast Please check the planning page if you are interested in participating with forecasts or advising on the project: http://ccmc.gsfc.nasa.gov/challenges/flare.php 1/4

2. Real time forecasting methods validation with the Flare Scoreboard Model questionnaire example 1.Is the forecast human generated, human generated but model based, model based, or other? 2.Does your flare prediction method forecast “M1.0-9.9” or “M and above”? 3.How do you specify active regions in your model? 4.Does your model provide uncertainties for the forecasted probability? If yes, what percentiles do you use to determine your upper and lower bound? 5.What prediction window(s) does your method use? 6.Do you have calibration for the probabilities from your model? If yes, what is considered a “low”, “medium”, or “high” level? Information about each model is needed in order to display only comparable methods together. This is gathered during the registration process. Please see our webpage for a more detailed questionnaire and information on how to register: http://ccmc.gsfc.nasa.gov/challenges/flare.php 2/4

3. Real time forecasting methods validation with the Flare Scoreboard Full-disk forecast submission example #File name format: Flare_Forecast_modelname_yyyymmdd_hhmm.txt Forecasting method: MAG4 Time: 2015-11-25T12:00Z Prediction Window Start Time: 2015-11-25T12:00Z Prediction Window End Time: 2015-11-26T12:00Z Probability Bins: M+ Input data: SDO/HMI LOS_Magnetogram Prediction window (hours): 24 #Full Disk Forecast #X_prob X_uncertX_LevelM_probM_uncertM_LevelC_probC_uncertC_Level #0.40000.0800 3 0.6800 0.0500 3 0.75000.05003 *_prob: Probability of X, M, or C class flare in decimal format (4 places) *_uncert: Uncertainty in X, M, or C class flare probability in decimal format (4 places) (optional) *_level: Calibration of probability for the model for X, M, or C class flares (1=low, 2=medium, 3=high) http://ccmc.gsfc.nasa.gov/challenges/flare.php A standardised.txt or.xml file can be automatically uploaded to CCMC via anonymous ftp, or manual form input used. Forecasts are parsed by the system every 6 hours. 3/4

4. Real time forecasting methods validation with the Flare Scoreboard 4/4 http://ccmc.gsfc.nasa.gov/challenges/flare.php Share your ideas with us on how to best display forecasts side by side, or how to combine forecasts: