© NERC All rights reserved Update on MMA, QL and AUX_OBS products Susan Macmillan, Brian Hamilton, Alan Thomson Various observations on 1st year of MMA, QL and AUX_OBS MMA: 90 minute cadence degree 1 magnetospheric vector field QL: Provides easy checking of Swarm data quality & some daily insight into hardware and processing issues AUX_OBS: Magnetic observatory database, updated every 3 months with quasi-definitive and definitive data INTERMAGNET developments: real-time and 1-second data Recent discussions on a 1-Minute Observatory data product

MMA processing to date Currently generating > minute models in two year-files: SW_OPER_MMA_SHA_2F_ T000000_ T000000_0101 SW_OPER_MMA_SHA_2F_ T000000_2014****T******_0101 Input data & models: Level1b Swarm ABC 0301 & CHAOS4+V4 for core field-model (with predictive fields to ). AUX_LIT for lithospheric field. CM5 ionospheric model (with F10.7 currently downloaded from NGDC) Processing: Select mid-/low-latitude Swarm data. Remove estimate of core-, lithospheric, and ionospheric-fields. Fit degree 1 internal and external model per orbit in GEO frame: Order 0 (dominant) coefficient by inversion Order 1 coefficients by mean-over-orbit Separate using 1-yr Q-response filter based on 1-D conductivity (Utada et al, 2003)

Comparison with Est & Ist (from Dst) Est & Ist (mean=0): Derived by NGDC Using Nov 2013 – Nov 2014 Using version with mean Ist = 0 Fairly good agreement with MMA: Compare in MAG frame Est: Correlation coeff = 0.84 RMS diff = 11.2 nT Ist: Correlation coeff = 0.90 RMS diff = 2.2 nT But some step-like changes (highlighted) Not clear yet why this is happening: diagnostics of MMA don’t indicate an obvious problem probably Dst baselines Year MMA - Est Est (nT) -MMA q10 (nT)

Comparison with VMD – order 0 coeffs VMD index (Thomson & Lesur, 2007): Minute-mean, mid-low latitude obs data Long-term trends subtracted 20 minute cadence. Internal & ext. deg 1 coeffs in GEO frame. Currently computed to Aug 2014 MMA-VMD agreement good for dominant order 0 coeffs: Compare in GEO frame q10 coeff: Correlation = 0.94, RMS = 13.9 nT (mostly offset) g10 coeff: Correlation = 0.91, RMS = 1.5 nT Any step changes in difference with VMD not so obvious, compared to difference with Est – better baseline control in VMD Year Steps? MMA - Est MMA - VMD VMD q10 MMA q

Comparison with VMD – order 1 coeffs Much poorer agreement between order 1 coeffs: CoeffCorrelation MMA-VMDRMS (nT) q11/s110.4/0.64.1/4.3 g11/h110.4/0.61.4/1.3 Comparison between MMA and TDS-1 during testing was much better. (corr coeff >~ 0.95) Scale of differences vary periodically: Local-time dependence? Noon-midnight orbits worst? MMA uses all LTs + ionospheric model: TDS-1 testing => better than night-only (Hamilton, 2012) But ionospheric model now used for real data RMS differences between model & input magnetospheric field residuals follow same pattern: Trying to fit > degree 1 fields? E.g. Ionospheric, magnetospheric higher degree fields RMS model-data Swarm A & B LTs MMA - VMD VMD q11 (nT) MMA q11 (nT) Year

Fields from degrees > 1 MMA fits degree 1 fields only For order 1 terms, degrees >1 may dominate (e.g. Lühr & Maus, 2010) Plot below shows magnetospheric residual data (black) and degree 1 fit (red): Orbit over night-side half of orbit on a quiet day from all 3 Swarm Clearly more structure than degree 1 field can fit Structure may be local-time dependent ~ 45 minutes

Satellite separation Any evidence of Swarm A and B seeing asymmetries in field due to LT dependence & FACs (e.g. Balasis et al., 2004)? Swarm A & B orbits now ~15 deg separated Re-ran MMA but using either Swarm A or B only Look for changes in differences as orbits diverge: Caused by differences in fields sampled Apparent overall growth in differences Could be due to LT-dependent field structure Results should be clearer as satellites further diverge and main field model updated Year s11: MMA A – MMA B q10: MMA A – MMA B Swarm A & B LTs

MMA Summary Robust daily automatic operation Dominant order 0 terms seem fairly robust (good agreement with VMD and Dst) Order 1 terms do not look very robust – possible day-side ionospheric and/or magnetospheric contamination? No clear longitudinal structure visible in Swarm A vs Swarm B models but still early in mission References Utada, H., T. Koyama, H. Shimizu, and A. Chave, A semi-global reference model for electrical conductivity in the mid-mantle beneath the north Pacific region, Geophys. Res. Lett., 30(4), 1194, Hamilton, B., Rapid modelling of the large-scale magnetospheric field from Swarm satellite data, Earth Planets Space, 65, 1295–1308, Luhr, H. and S. Maus, Solar cycle dependence of quiet-time magnetospheric currents and a model of their near-Earth magnetic fields, Earth Planets Space, 62, 843–848, Balasis, G., G. Egbert, and S. Maus, Local time effects in satellite estimates of electromagnetic induction transfer functions, Geophys. Res. Lett., 31, 16, L16610, 2004.

© NERC All rights reserved Daily quick-look products MAG-QL and EFI-QL process automatically daily at BGS MAG_QL example plots from a recent day Just over 1 hour apart in LT

© NERC All rights reserved Times series plots Swarm C scalar data ends late in the day (AUX_model is CHAOS4+V4, anomaly plot will look very similar for the official AUX model)

© NERC All rights reserved Example indicating problems… Problem (apparently GPS-related) not obvious from time series plot but now obvious in anomaly plot

© NERC All rights reserved EFI_QL example plots Recommended in should these be updated?

© NERC All rights reserved EFI_QL example B Thermal Ion Imager (TII) flag Langmuir Probe (LP) flags LP TII

© NERC All rights reserved AUX_OBS Selected observatory hourly means in geocentric coordinates submitted to PDGS All observatories together in individual year files

© NERC All rights reserved AUX_OBS Data are initially close-to-definitive data INTERMAGNET quasi-definitive data (data produced within 3 months of acquisition with accuracy close to that of definitive data) good quality data from other observatories produced in a timely manner. Accounts for 10-20% of the data in practice: almost-final baselines from manual measurements applied to cleaned variometer data and data released in a timely manner In time data are replaced with definitive data from World Data Centre database All data checked for quality by method described in Macmillan & Olsen, EPS, 2013

© NERC All rights reserved Residuals/measurement artefacts after initial selection At high latitude… and at mid-latitude… Quality checking

© NERC All rights reserved INTERMAGNET - Update Transmission of reported (raw) one- minute data within 72 hours to INTERMAGNET GIN Submission of one-minute definitive data at the end of each calendar year, meeting data quality specifications: Absolute accuracy: ±5 nT Resolution: 0.1 nT Band pass: D.C. to 0.1 Hz Inclusion of metadata; instrument baseline file, annual means file, observatory readme file Use of INTERMAGNET data exchange formats Full specifications documented in the INTERMAGNET Technical Manual

© NERC All rights reserved INTERMAGNET Current Status Observatory one-minute network Currently 125 INTERMAGNET Observatories (up from 120 in 2013) Quasi-definitive data Available from 61 observatories as of Sep 2014 (up from 51 in 2012, 61 in 2013) One-second data Available from 62 observatories as of Sep 2014 (up from 60 in 2012, 62 in 2013) Ongoing experimental exchange of non-definitive one-second data Real-time data exchange Target to reduce lag in collection and dissemination of real-time data to < 15 minutes Web interface introduced for rapid upload to INTERMAGNET GIN Improvements made in the exchange of data from GIN to INTERMAGNET web site New data format Draft CDF format designed to readily exchange small packets of data This common use format is also intended to exchange data (including metadata) for means: one-minute, hourly, daily, etc

© NERC All rights reserved The data should be close to the expected definitive value, but is to be delivered more rapidly than an observatory’s annual definitive data QD data are (H, D, Z) or (X, Y, Z) 1-minute data: corrected using temporary baselines made available less than three months after their acquisition; and such that the difference between the quasi-definitive and definitive (X, Y, Z) monthly means is less than 5nT for every month of the year (this can be checked a posteriori by comparing QD and definitive data from the previous year) Exchange mechanisms and formats now defined INTERMAGNET Quasi-definitive Data

© NERC All rights reserved Port Stanley Observatory QDD 2013

© NERC All rights reserved Improved Observatory Data for Swarm Science? Discussions on a 1-minute dataset of global geomagnetic observatory data (ESA, DTU, BGS) To aid in analysis/removal of rapid signals Magnetosphere, Lithosphere Would represent the ‘next level’ in observatory support for the Swarm mission (e.g. Magsat era = annual means; Oersted and Champ era = hourly means; Swarm era =...) CDF format with one file per observatory per year Hosted at WDC Edinburgh and/or ESA Swarm server

© NERC All rights reserved An observatory minute mean data product? Absolute observatory minute means consistent with the hourly means can be made available geocentric, measured discontinuities applied, series split if discontinuity not measured Separate files including essential metadata for each observatory- year, probably in CDF CDF 3-dimensional double precision zVariable for B, timestamp using TT2000 integer, metadata as a global attribute Initially concentrate on Swarm era, then progress to Champ (e.g. to improve crustal field recovery) Files to be made available every 3 months via ESA(?) and BGS servers 1-second data could also be made available for consistency with satellite magnetic field sampling