1. Grid Analysis of Radiological Data
Cécile Germain-Renaud
Laboratoire de Recherche en Informatique
Laboratoire de l’Accélérateur Linéaire
CNRS and Université Paris-Sud
For the AGIR team

2. LIMSI – coll Tenon, FMP, Sainte Anne
Contexts
A multidisciplinary research network
Funded by programme Masses de Données of the french ministry of research - 3 years from Sept. 2004
Medical image processing and computational/storage grids
LRI – coll LAL
LIMSI – coll Tenon, FMP, Sainte Anne
AlGorille
CRAN
LPC
CHRU Clermont
CNRS-STIC
CNRS-IN2P3
INRIA
INSERM
Hospitals
CREATIS
Rainbow
Epidaure
Centre Antoine Lacassagne
Journée GDR Stic Santé - 09/06/2005

3. Partners
Parallel Architecture - LRI – U. Paris-Sud & CNRS STIC Grid models & middleware
Cécile Germain-Renaud, Romain Texier
LIMSI – CNRS STIC Medical Image processing & software – Clinical research
Angel Osorio, Julien Nauroy and team, Emmanuelle Frenoux
Al Gorille – LORIA - U. Nancy & INRIA Lorraine Grid models & algorithms
Emmanuel Jeannot
CRAN – U. Nancy & CNRS STIC Image processing – compression
Jean-Marie Moureaux, Yann Gaudeau
CREATIS – CNRS STIC, INSERM Medical Image processing
Isabelle Magnin, Patrick Clarysse and team
LPC – U. Clermont & CNRS IN2P3 EGEE & Medical grids
Vincent Breton, Yannick Legré, Antoine Llorens
EPIDAURE - INRIA Sophia Image processing
Xavier Penned, Radu Stefanescu
RAINBOW-I3S CNRS STIC & U. Nice Software engineering – distributed components
Johan Montagnat, Tristan Glatard
Centre Antoine Lacassagne Clinical Research
Pierre-Yves Bondiau
Collaboration EGEE-LAL
Charles Loomis, Daniel Jouvenot
Journée GDR Stic Santé - 09/06/2005

4. The initial grid concept
1998 «  A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. »
Metacomputing(92), Iway(96)
Key ideas
Electrical power grid
Resource level /QoS /Ease of use
Journée GDR Stic Santé - 09/06/2005

5. Evolution of the Grid concept
“Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”
[I. Foster and C. Kesselman and S. Tuecke. The Anatomy of the Grid: Enabling Scalable Virtual Organizations. Intl Jal Supercomputer Applications,2002]
From computing to data
Institutional grids: computation on very large datasets
Particle physics production, Biomedical & Earth science deployed testbeds
Not much intersection with the semantic grid context so far
Resource sharing
Static: authentication, authorization, accounting
Dynamic: Scheduling
Journée GDR Stic Santé - 09/06/2005

6. Medical images and grids
LHC – PB/year: final decision for grid computing
ONE radiology department – 10TB/year
ONE CT-scan dataset 500MB
Standards for Integrated Data and Computational Grids are emerging: Global Grid Forum
Institutional support for operational deployement
EU FP6 project EGEE: 30M€ only for software development
US Grid3: 26 universities
Strong Computer Science research implication:
Structured grids: Globus
Unused cycles: Condor
CoreGrid
What is the relationship between MI and grids? Simply put, hospitals and all kind of medical centers produce much more data tahn even the future LHC. One single radiology department = 10TB/ year
Large Hadron Collider = 1PB/year
Why ? Because of all these scanners, RMI and so on, that are routinely used primarily for diagnostic but also intervention planning and research, especially in neuroscience.
Grids may be helpful with these large amount of information, first because of their raw power: storage and computation, and also because they provide the medium and softwware required to share them: this is the field of e-secinec, here e-medecine.
However, one single exam is by itself quite large: typically a DVD, and the grid may also help for everyday clinical practice,that is for problmes that you or me could face just tomorrow, and that is the subjetct of this demo.
At All levels of medical image analysis
Collaborations across widely distributed datasets
And/or Transparent access to large computing resources
Security/authentication infrastructure
Journée GDR Stic Santé - 09/06/2005

7. Objectives Leverage medical image algorithms and their usage
Integrated computational and storage grids as a unified resource provider for compute/data/network intensive needs
Image processing: research and clinical practice situations
Sharing data and algorithms
Specific grid services
Identification and definition: technology and research
Advanced middleware demonstrators
Impact grid projects
Strong collaboration with EU FP6 EGEE
Applications testbeds
Journée GDR Stic Santé - 09/06/2005

8. Algorithmic & Clinical research
Use cases
Epidemiology
Health care
policies
Medical collections
Ad Hoc Collections
Algorithmic & Clinical research
Algorithm research and deployment
Availability of a large panel of datasets and algorithms
Image guided diagnosis and surgical planning – Augmented reality
Transparent access to computing power
What is the relationship between MI and grids? Simply put, hospitals and all kind of medical centers produce much more data tahn even the future LHC. One single radiology department = 10TB/ year
Large Hadron Collider = 1PB/year
Why ? Because of all these scanners, RMI and so on, that are routinely used primarily for diagnostic but also intervention planning and research, especially in neuroscience.
Grids may be helpful with these large amount of information, first because of their raw power: storage and computation, and also because they provide the medium and softwware required to share them: this is the field of e-secinec, here e-medecine.
However, one single exam is by itself quite large: typically a DVD, and the grid may also help for everyday clinical practice,that is for problmes that you or me could face just tomorrow, and that is the subjetct of this demo.
At All levels of medical image analysis
Collaborations across widely distributed datasets
And/or Transparent access to large computing resources
Security/authentication infrastructure
e-Learning
Individual analysis
e-medicine
Augmented reality
Journée GDR Stic Santé - 09/06/2005

9. Work packages Medical Apps. Algorithm Gridification Core Grid Medical
Medical applications evaluation P-Y Bondiau
Medical
Apps.
Interactive
volume
reconstuction
A. Osorio
Cardiological
images
Segmentation
I. Magnin
Humanitarian
Medical
Development
V. Breton
Image
registration
in oncology
X. Pennec
Algorithm
Gridification
Workflow Management J. Montagnat
Dissemination C. Germain
Medical
data access
protocols
J-M. Moureaux
Services for
Interactivity
C. Germain
Medical data
Management
J. Montagnat
Core
Grid
Medical
Services
Middleware evaluation E. Jeannot
Journée GDR Stic Santé - 09/06/2005

10. Research areas (1) Scaling critical components of distributed systems
Scheduling: where and when ?
Agent-based scheduling and QoS: sharing resources across users
Workflow management: application performance
Contexts: (multi)processor scheduling, parallel scheduling
Medical data and metadata management (coll. LAL and MediGrid)
Interoperability between medical service/format standards and grid storage services
Security/privacy and medical requirements eg patient benefit
Adaptive storage and transmission
Compression methods
Network-adaptive compression (coll Network team CRAN)
User-adaptive compression: Intelligent remote data access - Data of Interest
Impact of network QoS (coll UREC)
Journée GDR Stic Santé - 09/06/2005

11. Research areas (2) Medical imaging algorithms
Parallel processing: 3D + time segmentation, non-linear registration
Bronze standard evaluation
Interaction between compression and image processing
Impact on intrinsic performance
Experiments: deployement and statistical analysis
Rationale ? (semantic) grid users
Integration of multi-scale/multi-level methods and compression
Journée GDR Stic Santé - 09/06/2005

12. Compression: QVAZM A Lattice Vector Quantization method
3D correlations: better detection of constant regions and contour preservation
Brain RMI 64:1 SPIHT 3D QVAZM
Brain RMI – 64:1 left SPIHT 3D right QVAZM
Journée GDR Stic Santé - 09/06/2005

13. Collection processing testbed
Grid-enabled
Workflow
PTM3D
measurements
Gold standard
Consensus
Bronze standard
Evaluation
Evaluation
Evaluation
PTM3D Volume
Reconstruction
Automatic
Nodules CAD
Registration
Algorithms
Network emulation
QVAZM3D Compression
Partially reliable
transport protocol
ADOC
SPIHT Compression
Journée GDR Stic Santé - 09/06/2005

14. The image database Goal: Provide an image dataset to the partners
Core: the personal collection of Angel Osorio - mainly CT
Centralised very simple SQL BD
Open to collaboration …
Journée GDR Stic Santé - 09/06/2005

15. Clinical situation testbeds (1)
gPTM3D: grid-enabling the PTM3D software
Focus: services for interactive grids - visualization pipeline - EGEE
Agent-based scheduling and QoS-oriented scheduling
Adaptive compression
Data and metadata management
Interaction
Render
Explore
Analyse
Interpret
Acquire
Journée GDR Stic Santé - 09/06/2005

16. Clinical situation testbeds (2)
CHINA – Collaboration between Hospitals for International Neurosurgery Applications
A web-based application
Goals
Enable medical data exchange :
Text-based data
Medical images
Second remote diagnosis
1 – YU Zhu Ping
Journée GDR Stic Santé - 09/06/2005

17. Services for Interactive Grids
Basic tools for seamless integration of the grid resources into general workflows
Convergence with non-biomedical applications
Depart from the computing centre model:
Local data or computations must be included
On-demand interactive
Id Owner Submitted ST PRI Class Running On
f01n    qzha     5/19 07:34 R  50  fewcpu       f11n07
f01n     agma    5/22 14:50 R  50  standard     f12n02
f01n     publ     5/22 16:16 R  50  standard     f03n09
f01n     agma    5/22 22:46 R  50  standard     f11n05
f01n     agma    5/23 12:41 R  50  standard    f01n11
f01n ying     5/23 14:17 R  50  fewcpu       f06n03
f01n     dpan     5/23 15:33 I  50  standard
f01n     divi     5/23 16:03 I  50  standard
f01n     publ     5/23 17:03 I  50  standard
f01n    publ     5/23 17:41 I  50  standard
f01n  dani     5/24 08:42 I  50  standard
GADU: on-demand
Algorithm research and deployment
Availability of a large panel of datasets and algorithms
Image guided diagnosis and surgical planning – Augmented reality
Transparent access to computing power
What is the relationship between MI and grids? Simply put, hospitals and all kind of medical centers produce much more data tahn even the future LHC. One single radiology department = 10TB/ year
Large Hadron Collider = 1PB/year
Why ? Because of all these scanners, RMI and so on, that are routinely used primarily for diagnostic but also intervention planning and research, especially in neuroscience.
Grids may be helpful with these large amount of information, first because of their raw power: storage and computation, and also because they provide the medium and softwware required to share them: this is the field of e-secinec, here e-medecine.
However, one single exam is by itself quite large: typically a DVD, and the grid may also help for everyday clinical practice,that is for problmes that you or me could face just tomorrow, and that is the subjetct of this demo.
At All levels of medical image analysis
Collaborations across widely distributed datasets
And/or Transparent access to large computing resources
Security/authentication infrastructure
Journée GDR Stic Santé - 09/06/2005

18. From PTM3D to gPTM3D
Volume and organs reconstruction help the surgeon to plan the optimal path
gPTM3D first result
interactive response time for volume reconstruction
Thus, the first goal of our work was to reach interactive response time for thsi task.
Journée GDR Stic Santé - 09/06/2005

19. Figures Dataset 87MB 210MB 346MB Input data 3MB 18KB/slice 9.6 MB
22KB/sclice
410KB
4KB/slice
Output data
6MB
106KB/slice
57MB
151KB/slice
86MB
131KB/slice
2.3MB
24KB/slice
Tasks
169
378
676 95
StandaloneExecution
5mn15s
1mn54s
33mn
11mn5s
18mn
36s
EGEE .
37s
18s
2mn30s
1mn15s
2mn03
24s
Small body
Medium body
Large body
Lungs
Journée GDR Stic Santé - 09/06/2005

20. Services for interactive grids
Front-end CE
EGEE
User Interface
Computational steering
Streams from/to a non-grid machine
to/from running applications
Coping with the submission penalty
N tasks each with small latency T
Potential completion bandwidth T-1
Impaired by the submission protocol
Application-level scheduling
Demonstrated at EGEE 1st and 2nd Conference - Selected as EGEE demo for the first EU review
[Germain, Osorio Texier. Interactive Volume reconstruction and measurement MIM 44(2)]
Interaction Bridge
Scheduling
Agent
Worker
Many applications, various solutions
Master/worker (push), coordinator/collaborator with various localisations of the « master »
Interact with remote data
Clinical research: evaluate registration algorithms on large existing databases –
Journée GDR Stic Santé - 09/06/2005

21. Scheduling for interactivity
Grid level scheduling cannot be centralized
Broker systems: matchmaking based on asynchronous information
Node schedulers can implement complex policies
Entities such as individual users, Virtual Organizations, applications, date,…
Past and current resource usage
Queue based: batch systems
Short Deadline Jobs
Immediate execution subject to access control without user reservation
No preemption/no utilization degradation
Context: Usage Policy Scheduling – Grid3
Fair share across SDJ: Soft real time scheduling with variable time quantum
Many applications, various solutions
Master/worker (push), coordinator/collaborator with various localisations of the « master »
Interact with remote data
Clinical research: evaluate registration algorithms on large existing databases –
Journée GDR Stic Santé - 09/06/2005

22. Interactive jobs on a grid: a scheduling problem
Short Deadline Job
A moldable application – individual tasks are very fine-grained
Soft deadline
No reservation: should be executed immediately or rejected
Sharing contract
Bounded slowdown for regular jobs
Do not degrade resource utilization
No stong preemption
Fair share across SDJ
Contexts
(multi) Processor soft real-time scheduling
Network routing Differentiated Services
To model these various interaction situation, we define SDJ: definition, example one ptm3d vol rec
Journée GDR Stic Santé - 09/06/2005

23. Scheduling SDJ Interaction Bridge Task prioritization TP
User Interface
User Interface
User Interface
Broker
Broker
Matchmaking
Sharing with batch users is simple. MAUI scheduler offers all possible configurations
Node
Permanent reservation
on virtual processors
Transparent when unused CE
Scheduler
Cluster
JSS
Journée GDR Stic Santé - 09/06/2005