1. Summary on EFI splinter
Igino Coco(1), on behalf of the Swarm EFI quality working group
ESA – Esrin, Frascati, Italy
6th Swarm Data Quality Workshop, Edinburgh, 29/09/2016

2. Summary Very fruitful and stimulating discussion (as usual)!
A large part of the discussion has been dedicated to data validation (Ne, Te, Electric field above all): techniques are mature, lot of work done, need to increase statistics and harmonize different approaches.
Progresses in the investigations of anomalies in Te: attempt to model and remove sweep hick-ups; spike trains vs Sun zenith angle still puzzling.
Impressive amount of work on-going on TII data:
Attempt to automate flagging of good/bad intervals basing on raw images properties;
Simplifying the processing concentrating on cross-track flow.

3. Data validation: comparison with other datasets approach; Swarm-ISR
initial
adjusted
By L. Lomidze

4. Data validation: comparison with other datasets approach; Swarm-ISR
initial
adjusted

5. Data validation: comparison with other datasets approach; Swarm-ISR
Measurements from LP, both high and low gain, seem to overestimate the electron temperature. “Quasi-linear” adjustments are proposed.
The electron density seems to be underestimated, in all S/C as well. Comparisons with radio-occultation measurements also confirm this picture.
Other previous results from Palin & Opgenoorth (IRF) roughly agree, but clearly high-latitude features are better evidenced.
Method by IRF is very robust in selecting conjunctions: only the very stable radar measurements in both space and time are used for comparison

6. Data validation: comparison with models (IRI)
Advantage: climatology of data calibration is given;
Drawback: a model is a model (good statistical information, but would it make sense to fit it with real data point by point?)
Proposed correction (for Te, same for Ne):
Pdiff = [(TeSW – TeIRI)/ TeIRI]% (known function of local time and latitude)
Tecorr = TeSW *100/(Pdiff+100)
Some evidence of agreement with C-NOFS measurements (Tion) when applying such correction
By V. Truhlik

7. Data validation: Electric Field and ionospheric conductances
Using two different methods for calculating the Pedersen currents (Spherical Elements Current System, Amm 1997; and empirical theory by Robinson et al., 1987), one can minimize a “Merit” function and obtain the “best” electric field values starting from initial values.
Event case, E from prel. dataset
Event case, E from oper. dataset
Initial
We already know OP dataset is not reliable for TII.
Nevertheless, the technique is able to “compensate” the initial values in the same way.
Corrected
By O. Marghitu

8. Summary and perspectives on validation
Proposed corrections to Ne basing on radio-occultation comparisons by two different authors:
S/C
N. Pedatella (2015)
L. Lomidze (2016)
Alpha
1.12 x *104
1.14 x *104
Bravo
1.31 x – 2.03*104
1.16 x *103
Charlie
1.08 x *104
1.13 x *104
Still too early for a definitive proposal of data correction.
Need to combine/increase statistics, and harmonize methodologies! Moreover corrections should be possibly applied as a function of latitude, local time, geomag. activity.
Other promising techniques have been proposed and deserve further exploitation: Ionosondes conjunctions with Swarm at particular conditions (or more systematic use of the ISR plasma line); density gradients from Swarm GPS (e.g. W. Miloch).
Comparisons with models (IRI, convection models…) are valuable, if complementary to other techniques, and could help to fill the measurements gaps.
Electric field validation by means of ionospheric conductance determination is very promising and deserves to be tested on more cases.
Extended dataset from IRF proved very useful: in general, the high gain probe seems to give more reliable Te measurements.

9. Electron temperature undesired features
ΔTe = A exp( (t-t0) / τ ) + C (t-t0)
Most of the hick-ups can be fitted by exponential functions, filtered and interpolated, but not all of them.
The decay time is inverse proportional to ambient Ne, the amplitudes of the “hick-ups” are inversely proportional to the decay times; the dependences change abruptly at terminator.
Amplitudes and decay times of the “hick-ups” seem to be smaller for the high-gain probe compared with the low gain probe, and the relationships are less distinct.
By M. Förster

10. Electron temperature undesired features
Recurrent “spike trains” (5000 K and above) are observed in the electron temperature as measured by the Swarm Langmuir probes. The distribution of such spikes shows a strong variability around the orbit and seems to be clearly correlated with the solar illumination.
The overplotted lines show the angle between the solar panel's normals and the sun direction (notified as "COS"-values in the upper left corner). The "Te spike trains" follow mostly quite closely these lines of ~ deg and ~88 deg angles.
Most of the large amplitude hick-ups that cannot be simply modelled occur in the same conditions

11. Electron temperature undesired features
spike trains are often associated with sharp transitions of the solar panel currents (red circled areas.
Not all the spike trains are associated to such transitions: some of them can well be “geophysical” rather than “instrumental” (blue circled area).
Analysis to be continued. Where does this illumination effect come from? Photoelectrons from the panels? Why LP should be so much affected?

12. Towards an automated flagging of TII data
Single Image Criteria:
Consider subset of pixels with relatively large (> 75%) intensities;
Distance between the center of the image and MCP Origin does not exceed 4 pixels;
Image effective radius does not exceed 4 pixels;
Total number of intense pixels > 5;
Images Sequence Criterion:
Consider sequence of consecutive (>=2) valid Images only
By A. Kouznetsov

13. New concept of TII production and distribution
L1b Prototype is complex to calibrate – simplify at expense of loss of some accuracy
Derive cross-track flows from flow angles, assuming 7.6 km/s ram speeds (Use reference frame co-rotating with Earth, remove flows arising from yaw and pitch motion; Calibrate cross-track horizontal flow against (co-rotation + yaw) signal; Exclude noisy data)
Send latest measurements to ESA regularly
Expert judgment needed: data will be first made available to expert users, with some basic documentation on how to use them, and how to orient the measurements given in instrument frame.
Eventually, good data will be automatically selected.