1. Combined use of AMDA, CAA, QSAS and CL based on time-table exchange. C. Jacquey, V. Génot, E. Budnik, M. Bouchemit, M. Gangloff, CDPP/CESR B. Cecconi, CDPP/LESIA C. Perry, CAA team T. Allen, A. Rochel, QSAS team E. Penou, J.A. Sauvaud, CESR Cluster Workshop, Tenerife, March 12, 2008 This talk is related to: -Poster (Jacquey et al.) this evening -Talks on CAA (C. Perry), QSAS (T. Allen) and CL (E. Penou)

2. Some aspects of the studies of space physics ·We need to analyse measurements obtained by several observatories ·We need to analyse multi-instrument data ·We need to compare the observations to simulation results We need to extract and treat a lot of data coming from multiple resources  This takes time and energy ·Dynamical processes  The time is the very first parameter ·The studies consists of analysis/characterisation of events treated in case studies or in statistical studies · Creating Time-Tables (or event list) is a common step of the work

3. Time-Tables (Event Lists)  Time-Table produced by the users  Time-Tables coming from statistical studies (e.g. magnetopause crossing lists) sometimes made available to the community (e.g. the Frey’s substorm list)  Time-Table coming from search tool based on orbitographic or conjonction criteria (e.g. the Query tool of the SPDF/SSCWeb)  Time-Table coming from search tool based on mathematical criteria applied to the content of the data (e.g. AMDA of CDPP)  Time-Table coming from visual search tool (e.g. AMDA of CDPP) Exchanging Time-Tables between data centres, services and tools?  saving time and energy. Increase of the scientific return.

4. AMDA Automated Mutiple Dataset Analysis  Web based service  Transparent access to data  AMDA local database (CLUSTER, ACE, THEMIS, GEOTAIL, WIND, …, ISEE)  External database (CDAWeb, MAPSKP, THEMIS/CESR, …)  Produces and exploits time-tables Visualisation editor Parameter editor Download data Conditional search External data Visual search Time-Table manager AMDA DOES NOT WORK WITH InternetExplorer and Safari AMDA is public (http://cdpp-amda.cesr.fr)

5. Others DARTS SSL THEMIS database Others … SPDF SSCWeb  Time-Tables + many services GAIA Virtual Observatories VSPO, VMO, VHO, VIRBO, … CL QSAS Resources: data centres, services, tools CAA Validated high resolution CLUSTER data Plots and tools Mirror THEMIS database CDPP database AMDA  Time-Tables CDPP CDAWeb

6. Science Case: study and characterisation of the local properties of the cross-tail current  Based on CLUSTER data when it encircles the neutral sheet. (i)Current density, profile, gradient (ii)Thickness, location, motion (iii)Distribution function (i+ and e-), flows, … (iv)Turbulence/wave level and spectra (v)…  Influence factor: (i)Solar Wind pressure (ii)IMF (iii)Geomagnetic activity: -State of the ionosphere. -Oxygen abundance -Ring current intensity

7. Use Case: the main steps Access and extraction of ACE/WIND/GEOTAIL Interplanetary data Access and extraction of ground (AE, AL, SYM, DST) data Access and extraction of modelisation or simulation data (Tsyganenko, CCMC, …) Selection of the periods of NS encircling condition User tools Basic treatments Formatting Data merging Time shifting Results Catalogues Maps Models Selection of the periods of NS encircling condition AMDA Access and extraction of ground (AE, AL, SYM, DST) data Access and extraction of ACE/WIND/GEOTAIL Interplanetary data CDAWeb, CDPP, AMDA, DARTS, … CAA Access and extraction of CLUSTER data Access and extraction of CLUSTER data Specific tools Specific tools CL Generic tools Generic tools CL, QSAS or AMDA

8. Creation of time-tables on AMDA/CDPP ÞTT1, time-table listing the periods when the conditions are fulfilled ÞTT2, extended time-table

9. Conditional search

10. Visualisation

11. Visual search

12. Managing Time-Tables Text or XML VOTable

13. Parameter editor

14. Data dowload Data merging

15. AMDA -Automated search -Visual search  Time-Tables CAA Validated high resolution CLUSTER data CL QSAS Standardised sub-databases Other? CDAWeb DARTS CDPP/CESR THEMIS database SSL THEMIS database Other? Time-Tables First step: producing sub-databases corresponding to the time-tables Time-Tables Data, parameters User tools

16. How to do that?  Standardisation of the time-tables  Standardisation of the organisation of the sub- databases to be produced  Interface implementation on CAA for reading the time- table and producing sub-databases  Interface implementation on the tools QSAS and CL  Standardisation of the time-table description

17. Step 2: catalogues T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … AMDA, SSCWeb or other services CAA, CDAweb, CDPP or other data centres QSAS, CL, AMDA or other tools

18. Time-table: T1 – T2 …. AMDA USER Computation of P1,..,PN CAA Catalogue: T1, T2, P1, …, PN Sub-database Catalogue: T1, T2, P1, …, PN, Q1, …,QN Future: Archiving the catalogues for availability to the community QSAS or CL CDPP CDAweb DARTS USER Catalogues and associated tools available at the data centres

19. Conclusion  Exchange of standardised Time-Tables between data centres, services and tools should allow the researcher - to save large amount of time and energy - to exploit more extended data amount - to increase the scientific return of the data  First step: extraction and exploitation of sub-databases corresponding to the Time-Tables  Second step: creation and exploitation of catalogues, similarly to astronomy and solar physics  This project starts with CAA, QSAS, CL and AMDA, but of course, other new participant is welcome  The specifications of this project need the input of the potential users  Poster this evening in order to collect suggestions and ideas. AMDA:contact:amda@cesr.framda@cesr.fr Stable version:http://cdpp-amda.cesr.frhttp://cdpp-amda.cesr.fr Test version:http://iapetus.cesr.fr/AMDA_PRETEST/http://iapetus.cesr.fr/AMDA_PRETEST/

24. Use cases Use case = simulation of studies which could be performed by the researchers in order to:  identify and analyse their needs  specify the user requirements  Imagine, define and specify the system to be developed The developers of the system need the input from the community Obvious requirement: standardisation  Standardisation of the data format ÞCEF, CDF, NetCDF, (FITS) ÞBut beyond that, standard way for representing data of different classes (particle, waves, Time-Tables, …)  Standardisation of the data description  SPASE (Space Physics Archive Search and Extract, http://www.spase-group.org/http://www.spase-group.org/ )

25. Step 2: sub-database extraction ÞOn CAA, CDAweb, DARTS, CDPP, AMDA, … Requirements: %Standard format for time-tables %Interface for reading and managing the time-tables %User interface for selecting data %Optionally, basic on the fly treatment (e.g. data merging, SW propagation shifting, …) %Standardised management of the multiple file output ·Filename ·Descriptor ·Product recovering

26. download

27. Pi-Pj/(Xi-Xj)

28. Step 3: plot production on CL, QSAS, AMDA %INPUT: ▪ time table ▪ corresponding sub-database %Manipulation of the time-table %User edited plot %Launch in batch mode Requirements:  Standardisation (i) of the time-table and (ii) of the data  Plot editor  Batch mode  Standardised management of the plot files ▪ Filename ▪ Descriptor ▪ Product recovering

29. AMDA Automated Mutiple Dataset Analysis Created by the CDPP: http://cdpp.cesr.fr/http://cdpp.cesr.fr/ Stable version: http://cdpp-amda.cesr.fr/http://cdpp-amda.cesr.fr/ Testing version: http://iapetus.cesr.fr/AMDA_PRETEST/http://iapetus.cesr.fr/AMDA_PRETEST/ Contact: amda@cesr.framda@cesr.fr

30. Example: event search Old fashion: “paper” search AE, ALISEE-data IMP-8 AE, AL ISEE MAG ISEE- electron IMP8 MAG IMP8 PLA Current fashion: web Event search takes time and energy CDAWeb SPDF UCLA CAA CDPP And others

31. Automated Mutiple Dataset Analysis AMDA: the goals Multiple Dataset: the study of the (multi-scale) dynamics of plasma objects requires to perform the integrated analysis of multi-point and multi-instrument data Automated (or semi-automated) Analysis: help for  Search, characterisation, classification of events  Extensive exploitation of the vast database (statistics)  “historical” studies (example: through solar cycles)  Building catalogues  Building virtual constellations  Time-Table (EventList) production

32. AMDA, fact sheet Built in IDL, C, FORTRAN, Javascript Works on standardised and “simple” data, NetCDF Includes models (Tsyganenko, Shock, MP, NS, …) Uses external data (now from CDAWeb and soon from MAPSKP, IWF, THEMIS, …) Forum for user feedback Twiki for information (development, bugs, …) exchange Already in use for scientific studies AMDA DOES NOT WORK WITH InternetExplorer and Safari AMDA is public

33. Functionalities  Automated access to data  Access to and on-the-flight ingestion of external data (test version only)  Data merging  Data download  Data visualisation  User edited parameter computation on the data  Automated conditionnal search  Visual search Production of Time-Tables

34. Local Database AMDA System CDAWeb CDPP MAPSKP AMDA/”External Data” others

35. Needed development on AMDA  Temporal functions (mean, regression, dispersion estimator, …)  User pluggins  Continuous SW propagation shifting  Scatter plots, map plots, …  Your suggestion  ….

36. Conclusion and questions  CAA, QSAS, CL and CDPP teams are starting a collaboration for providing to the researchers ways to save time and energy for performing their studies  Other services and database are invited to join this effort  We need input, comments, suggestions from the potential users.