A framework for the analysis of medical data using data sonification across the GÉANT network and the European Grid infrastructure Roberto BARBERA University of Catania & INFN CT, Italy Francesca FALCETTA The Mario Negri Institute for Pharmacological Research, Milan, Italy Massimo RIZZI The Mario Negri Institute for Pharmacological Research & ARCEM, Milan, Italy Giuseppe LA ROCCA INFN CT & IGI, Italy Mariapaola SORRENTINO ASTRA Project Domenico VICINANZA DANTE, Cambridge, UK

2 Connect | Communicate | Collaborate The Data Sonification process in a nutshell How does the sonification process work ? The Mario Negri Institute The Epilepsy disease – EEGs – Intracranial vs. conventional EEGs recording The mission The role of advanced R&E networking and Computing Infrastructures GEANT Network and the Catania Science Gateway The Sonification process running on a Science Gateway First preliminary results Summary & Conclusions Outline

3 Connect | Communicate | Collaborate  The Data Sonification process in a nutshell How does the sonification process work ? Benefits of the sonification techniques

4 Connect | Communicate | Collaborate Representation of data sets and information through sounds and melodies Data sonification Credits: D. Vicinanza et al. LHC Open SymphonyLHC Open Symphony Same as creating a graph using notes and tones instead of lines and points

5 Connect | Communicate | Collaborate How does sonification work? Associating a musical note to each value Two conditions: Uniqueness: – Same value  same note – Ex: 25  middle C Covariance: – Melody varies as quickly as the data. – Ex: 25  C, 26  D, 27  E …

6 Connect | Communicate | Collaborate From numbers to notes Correspondence between numbers and notes: Large numbers  high pitches Small numbers  low ptiches The same numbers are associated with the same notes

7 Connect | Communicate | Collaborate Example: periodic data

8 Connect | Communicate | Collaborate Periodic melody

9 Connect | Communicate | Collaborate Advantages of sonification Acoustic counterpart of data graphic visualization Sonic representations are particularly useful dealing with complex, high-dimensional data, data monitoring tasks pattern detection Transitions between states become audible Sound is not directional Sound can pack more information Human ear naturally trained to detect patterns and anomalies

10 Connect | Communicate | Collaborate Two main applications: Scientific provide alternative ways to analyse data Artistic write music inheriting from the richness of data – New inspirational ideas

11 Connect | Communicate | Collaborate  The Mario Negri Institute  What is Epilepsy ?  Epilepsy in numbers  EEGs  Intracranial EEGs recording  The mission

12 Connect | Communicate | Collaborate The Mario Negri Institute for Pharmacological Research Not-for-profit biomedical research organisation (est 1961) From the molecular level to the whole human being Developing new drugs and Making existing ones more effective Main research headings: Cancer research Nervous and mental illnesses Cardiovascular and kidney diseases Rare diseases Toxic effects of environmental contaminants

13 Connect | Communicate | Collaborate What is Epilepsy ? Epilepsy is a brain disorder that causes people to have recurring seizures Clusters of neurons, in the brain send out the wrong signals Epilepsy is an illness as old as human kind. It was mentioned in ancient Babylon more than 3,000 years ago Strange sensations and emotions or behave strangely. Violent muscle spasms, auras, convulsions or lose consciousness Many possible causes including illness, brain injury and abnormal brain development. In many cases, the cause is unknown

14 Connect | Communicate | Collaborate Epilepsy in numbers Still an unresolved problem Still one of the most common neurological disorder ~ 1% of the world-wide population affected 50 million people – 90% in developing countries More likely to occur in young children or over 65 ~ 25% of patients not responding to conventional drugs Famous people affected by epileptic seizures: Vincent van Gogh Julius Caesar Alexander the Great Lord Byron

15 Connect | Communicate | Collaborate EEGs An electroencephalogram (EEG) is a test used to record the brain’s electrical activity From EEGs physicians can look for abnormal patterns: seizures encephalopathies Red-colored area highlights the abnormal EEG activity peculiar of a seizure Usually, the onset of a seizure is associated to a rapid build up of 4 to under 8 Hz rhythmic activity of EEG. Diagnose: Brain scans, EEGs…

16 Connect | Communicate | Collaborate Intracranial EEGs recording Usually recorded from scalp, also from electrodes within the brain - intracranial EEGs recording less affected by spurious signals Electrode strip Source: Freiburg’s EEG database

17 Connect | Communicate | Collaborate The mission “inter-ictal” EEGs – EEGs recorded sufficiently far from seizures Analysed for the first time using a sonification technique – to identify a baseline condition (hopefully, a marker) This may help to: highlight and characterize patterns provide a powerful tool for seizure prevention  by listening to how musical patterns change  forecast an impeding seizure in advance. promote the investigation for new therapeutic interventions

18 Connect | Communicate | Collaborate  The role of advanced R&E networking and Computing Infrastructures  The Sonification process running on a Science Gateway The Catania Science Gateway components

19 Connect | Communicate | Collaborate The role of advanced R&E Networking and Computing Infrastructures Sonification and subsequent data analysis requires reliable networks and processing power to produce results –To analyse just a few MB of data using, for example, 100 different configurations would require a few hours on a standard laptop (2.4 GHz Intel Core 2 Duo, 2 GB RAM) and just a few minutes on the grid –EEG sequences needs to be analysed using different parameters and different mapping options –High quality audio files generated and analysed for each EEG data sequence For this analysis we used the MidiToolbox software Other tools are under investigation (i.e. WaveLab)

20 Connect | Communicate | Collaborate Bringing grid computing in hospitals Hospital Grid portal Grid infrastructure + NRENs Grid portal based on Science Gateway allows researchers in the hospital to access grid services through GEANT

21 Connect | Communicate | Collaborate The GÉANT Network ( Connecting EU NRENs GÉANT is co-funded by Europe’s NRENs and the European Commission (EC) under the Seventh Framework Programme (FP7) Project Partners are 32 European NRENs, TERENA and DANTE 150 FTEs’ annual effort (> 350 individuals) working in GÉANT across Europe 25 European POPs 12,000km of dark fibre on 18 routes 50,000km network infrastructure on 44 routes Diversified footprint Serves 40 million users 8,000 institutions Across 40 European countries Courtesy of T. Fryer

22 Connect | Communicate | Collaborate GÉANT: Global Reach GÉANT connects to 62 countries outside Europe

23 Connect | Communicate | Collaborate The Science Gateway Web-based portal to grid services Nothing to install Based on standards Modular Easy to use Re-usabile Useful to offer grid access anywhere The architecture: JSR 286 standard and Liferay as portlet container Identity Federations based on the SAML 2.0/Shibboleth Computing resources access: (J)SAGA

24 Connect | Communicate | Collaborate Grid transactions Grid transactions: Robot X.509 certificates JAX-RS and PKCS#11 standards Compliant with: EGI VO Portal Policy EGI Grid Security Traceability and Logging Policy

25 Connect | Communicate | Collaborate Science Gateway Science Gateway Standard-based (SAGA) middleware-independent Grid Engine The Catania Science Gateway Components Users from different organizations having different roles and privileges Administrator Power User Basic User

26 Connect | Communicate | Collaborate Access the data Sonification process It provides instructions for users and configuration settings running on

27 Connect | Communicate | Collaborate Select the computing resource running on It shows the computing resources where the SW has been successfully deployed It is possible to select & rank the generic computing resource

28 Connect | Communicate | Collaborate running on Upload the data set as ASCII file or using the text-area Configure some advanced settings Job description Enable notification Select the type of algorithm to be used Enable the analysis of MIDI file with Matlab Select the data set

29 Connect | Communicate | Collaborate Check the status of the simulations Asks for support Automatic jobs output downloading Shows the status of the simulations

30 Connect | Communicate | Collaborate Bringing grid computing in hospitals Hospital Grid portal Grid infrastructure Thanks to the Grid portal data is sent back to the researchers for analysis + NRENs

31 Connect | Communicate | Collaborate  First preliminary results EEG analysis: tonal map EEG analysis: melodic event series EEG analysis: melodic contour EEG analysis: interval distribution  Summary & Conclusions  Links & Contacts

32 Connect | Communicate | Collaborate EEGs analysis: Tonal Map Frame extracted from a tonal map animation showing the evolution of the melody around a tonal centre. The idea: correlate tonal distribution with the disease stages to find a marker

33 Connect | Communicate | Collaborate EEGs analysis: Melodic Event series The series of music events (notes) as a function of time. The ultimate goal is to contribute to the determination of an actual music marker The idea: find clustering and changes in the distribution and correlate them with phases or stages in the disease

34 Connect | Communicate | Collaborate EEGs analysis: Melodic Contour The series of music events (note pitches), connected by line segments, as a function of time. The idea: find clustering and changes in the distribution

35 Connect | Communicate | Collaborate EEGs analysis: Interval Distribution The distribution of music intervals in the analysed EEGs Here as well, the hope is to The ideafind the proper mapping so to correlate that distribution with the disease stages to find a marker

36 Connect | Communicate | Collaborate Summary and Conclusions Sonification techniques are the most appropriate tools for extrapolating temporal-correlated properties in time series as musical patterns Advanced R&E networking and e-Infrastructures have been used to identify “markers” for denoting some specific epileptic EEG states Our research is at early stage.  Only few EEGs have been analysed.  More research is needed to corroborate preliminary results!  Visual approaches don’t take temporal information into account

37 Connect | Communicate | Collaborate Download the Sonification White paper (PDF)White paper (PDF) EEG Sonification For Epilepsy Surgery: A Clinical Work-In Progress [link]link Midi Toolbox [link]link Contacts: Roberto Barbera Francesca Falcetta Giuseppe La Rocca Massimo Rizzi Mariapaola Sorrentino Domenico Vicinanza

38 Connect | Communicate | Collaborate | | Connect | Communicate | Collaborate Thank you!