1. Liceo Scientifico Statale
Erice – December, 06-08, 2017
ADM: Automatic Data Miner: a customizable tool for selecting, merging and analysis of the EEE trending data sets
Liceo Scientifico Statale
Piero Gobetti - Torino
Lorenzo Sibille – Lorenzo Mathis

2. Data from the over 56 telescopes all over Italy are sent to CNAF-INFN, in Bologna, to allow track reconstruction so that all relevant information can be stored in a database to be later available for analysis.

3. Data for schools are available in csv format at http://eee.centrofermi.it/monitor

4. Choose the telescope
Select the day(s)
And download data in csv format

5. Problems: daily DQM report contains redundant data;
you have to erase superimposing periods
Data overlapping:
Data loss:
sometimes data are lost due to telescope malfunction or telescope switched off
Result: time waste in choosing another data period and long data formatting time

6. Solution: Automatic Data Miner
- choose telescope and data period
concatenates many csv data file from chosen telescope
and period
- erase superimposing periods
- make data consistency check (data loss)
output in csv, MySQL, HTML or Excel™ format

7. Automatic Data Miner
Web Interface

8. Automatic Data Miner a) Creates a /tmp/usr_x directory
Web Server
(PHP reads query)
HTML5 + CSS3 + JS(JQuery)
AJAX Post()
Python
a) new_user
b) downloader
c) data_unit
d) data_check
a) Creates a /tmp/usr_x directory
CNAF-INFN csv database
b) Downloads and concatenates .csv files
c) Deletes superimposing records
d) For each record checks if timestamp(n+1)-timestamp(n) > MAX_DATA_LOSS

9. Automatic Data Miner
Lost data: 86 min
Lost data: 73 min
Lost data: 73 min

10. Further implementations
FATA: Forbush Automated Tool for Analysis
Forbush decrease ( )
We’ll publish ADM on our website and we’ll make ADM available to CNAF-INFN EEE Monitor web site
FATA will be developed also looking at future Centro Fermi - EEE (TO group) collaboration with INAF-OATO
(Osservatorio Astronomico di Torino) Space Weather
project

11. Dr. Ivan Gnesi - Centro Fermi Dr. Fabrizio Coccetti - Centro Fermi
Many thanks to:
Dr. Ivan Gnesi - Centro Fermi
Dr. Fabrizio Coccetti - Centro Fermi
Dr. Alessandro Bemporad – OATO
And many thanks to everybody at Centro Fermi!!
Questions and info: Fulvio Poglio,