Coordinated Ionospheric Model Testing Sean Elvidge, Matthew Angling November 25th, 2015

Need for Common Testing Scenarios The comparison of models to observations is commonplace throughout the sciences. Comparing models allows better understanding of the success of different methods and techniques. Existing multi-model comparisons include Shim et al. (2011, 2012) Eight models tested Multiple scenarios Short test scenarios Feltens et al. (2011) Five models compared Two scenarios Propose a community wide, long term (30 day), test scenario Performance not too bad (comparable to IRI) – thesis (mTd image) 2 2

December 8th 2008 – January 7th 2009 Ionosonde: Chilton (RL052) Juliusruh (JR055) Pruhonice (PQ052) Dourbes (DB049) GPS: BOR1 BRUS GOPE HELG HERT OPMT POTS PTBB WROC WSRT ZIMM REDU 3

December 8th 2008 – January 7th 2009 4

Testing Parameters Initially plan to test NmF2 hmF2 h(0.64NmF2) (height at 64% of NmF2) A less noisy measurement parameter than hmF2 dSTEC (differential slant TEC) Temporal and spatial test Typically more accurate than ~0.1 TECU Allows the testing of models which do not give full profile shape Is there something else you would like to see tested? 5

Data Locations & Procedures Ionosonde data should be downloaded from NGDC (ftp.ngdc.noaa.gov) GPS data should be downloaded from SOPAC/CSRC archive (garner.ucsd.edu) All of this information, including Python code to download data is at: http://tinyurl.com/testscenario If you are interested in performing the test: Working Meeting: Ionospheric effects on Radio Systems (Thursday 1500 – 1630, Delvaux) Contact: s.elvidge@bham.ac.uk 6

Current Status of Test Models being tested include At University of Birmingham AENeAS EDAM (QinetiQ) TIE-GCM (NCAR) GITM (University of Michigan) CCMC (Run on request: AT-RASC) CTIPe USU-GAIM External partners NeQuick (with and without assimilation, ICTP) IRI (UMASS Lowell) IRTAM (IRI with assimilation, UMASS Lowell) Next presentation (Ivan Galkin @ 1140) UQRG (UPC) 7 7

NeQuick - ICTP Bruno Nava 8

NeQuick with Data Assimilation (Stage 1) To incorporate ionosonde information Effective parameters are generated such that modelled peak parameters (foF2 / hmF2) match observations An updated background model is obtained which is used for TEC assimilation. Performance not too bad (comparable to IRI) – thesis (mTd image) 9 9

NeQuick with Data Assimilation (Stage 2) A Best Linear Unbiased Estimator (BLUE) is used to assimilate ground-based TEC data into NeQuick Calibrated slant TEC data is used for the assimilation. "Simple" observation and background error covariance matrices are used Performance not too bad (comparable to IRI) – thesis (mTd image) 10 10

UPC Tomographic-Kriging GIMs Manuel Hernández-Pajares, Alberto García-Rigo 11

UPC Tomographic-Kriging GIMs (UQRG) Performance not too bad (comparable to IRI) – thesis (mTd image) Hernandez-Pajares, M., Juan, M. and Sanz, J., 1999. New approaches in global ionospheric determination using ground GPS data. Journal of Atmospheric and Solar-Terrestrial Physics 61, pp. 1237–1247. 12 12

dSTEC Std. Dev. Error with respect to REDU REDU not included in construction of UQRG model 13 13

dSTEC Bias with respect to REDU REDU not included in construction of UQRG model 14 14

Conclusions Test scenario proposed at AT-RASC, May 2015 Long, quiet time test Currently being run, or has been run, for 11 different models Wider community participation encouraged New physics-based data assimilation model being developed at UoB (Sep slide) Time on x axis, annotation for ntrip data etc.etc. 15 15

Future Talks / Meetings Ivan Galkin @ 1140 (next presentation) – IRTAM Results Working Meeting: Space Weather Metrics, Verification and Validation (Today 1500 – 1630, Delvaux) Working Meeting: Ionospheric effects on Radio Systems (Thursday 1500 – 1630, Delvaux) Beacon Satellite Symposium 2016, Trieste, Italy June 27 to July 1, 2016 Session on 'Data Assimilation' Results of test scenario (Sep slide) Time on x axis, annotation for ntrip data etc.etc. 16 16