Enabling Grids for E-sciencE EGEE-II INFSO-RI-031688 Medical Data Manager 1 Dicom retrieval : overview of the DPM One command line to retrieve a file:

Slides:



Advertisements
Similar presentations
Workflows over Grid-based Web services General framework and a practical case in structural biology gLite 3.0 Data Management David García Aristegui Grid.
Advertisements

FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.
John Degenhart Joseph Allen.  What is FTP?  Communication over Control connection  Communication over Data Connection  File Type  Data Structure.
Makrand Siddhabhatti Tata Institute of Fundamental Research Mumbai 17 Aug
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
EGEE-II INFSO-RI Enabling Grids for E-sciencE gLite Data Management System Yaodong Cheng CC-IHEP, Chinese Academy.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America The AMGA metadata catalog with use cases.
INFSO-RI Enabling Grids for E-sciencE gLite Data Management Services - Overview Mike Mineter National e-Science Centre, Edinburgh.
INFSO-RI Enabling Grids for E-sciencE Logging and Bookkeeping and Job Provenance Services Ludek Matyska (CESNET) on behalf of the.
EGEE-III INFSO-RI Enabling Grids for E-sciencE The Medical Data Manager : the components Johan Montagnat, Romain Texier, Tristan.
INFSO-RI Enabling Grids for E-sciencE Project Gridification: the UNOSAT experience Patricia Méndez Lorenzo CERN (IT-PSS/ED) CERN,
The LCG File Catalog (LFC) Jean-Philippe Baud – Sophie Lemaitre IT-GD, CERN May 2005.
INFSO-RI Enabling Grids for E-sciencE Distributed Metadata with the AMGA Metadata Catalog Nuno Santos, Birger Koblitz 20 June 2006.
Author - Title- Date - n° 1 Partner Logo WP5 Summary Paris John Gordon WP5 6th March 2002.
INFSO-RI Enabling Grids for E-sciencE AMGA Metadata Server - Metadata Services in gLite (+ ARDA DB Deployment Plans with Experiments)
Enabling Grids for E-sciencE EGEE-III INFSO-RI I. AMGA Overview What is AMGA Metadata Catalogue of EGEE’s gLite 3.1 Middleware Main Feature of.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks JRA1 summary Claudio Grandi EGEE-II JRA1.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE middleware: gLite Data Management EGEE Tutorial 23rd APAN Meeting, Manila Jan.
Enabling Grids for E-sciencE Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks AMGA PHP API Claudio Cherubino INFN - Catania.
INFSO-RI Enabling Grids for E-sciencE OGSA DAI Data Access and Integration Marek Ciglan Institute of Informatics, Slovac Academy.
Enabling Grids for E-sciencE Jean Salzemann EGEE08 Conference – Istanbul – 25/09/08 1 HOPE HOsptital Platform for.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Hands on session: the AMGA Metadata Catalogue.
INFSO-RI Enabling Grids for E-sciencE gLite Data Management and Interoperability Peter Kunszt (JRA1 DM Cluster) 2 nd EGEE Conference,
EGEE User Forum Data Management session Development of gLite Web Service Based Security Components for the ATLAS Metadata Interface Thomas Doherty GridPP.
INFSO-RI Enabling Grids for E-sciencE Experiences with LFC and comparison with RNS Erwin Laure Jean-Philippe.
From Digital Objects to Content across eInfrastructures Content and Storage Management in gCube Pasquale Pagano CNR –ISTI on behalf of Heiko Schuldt Dept.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Site Architecture Resource Center Deployment Considerations MIMOS EGEE Tutorial.
INFSO-RI Enabling Grids for E-sciencE Αthanasia Asiki Computing Systems Laboratory, National Technical.
WP 10 ATF meeting April 8, 2002 Data Management and security requirements of biomedical applications Johan Montagnat - WP10.
Managing Data DIRAC Project. Outline  Data management components  Storage Elements  File Catalogs  DIRAC conventions for user data  Data operation.
SEE-GRID-SCI Storage Element Installation and Configuration Branimir Ackovic Institute of Physics Serbia The SEE-GRID-SCI.
INFSO-RI Enabling Grids for E-sciencE Introduction Data Management Ron Trompert SARA Grid Tutorial, September 2007.
Database authentication in CORAL and COOL Database authentication in CORAL and COOL Giacomo Govi Giacomo Govi CERN IT/PSS CERN IT/PSS On behalf of the.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Data management in LCG and EGEE David Smith.
INFSO-RI Enabling Grids for E-sciencE /10/20054th EGEE Conference - Pisa1 gLite Configuration and Deployment Models JRA1 Integration.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks gLite Data Management Components Presenter.
EGI-Engage Data Services and Solutions Part 1: Data in the Grid Vincenzo Spinoso EGI.eu/INFN Data Services.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Storage Accounting for Grid Environments Fabio Scibilia INFN - Catania.
INFSO-RI Enabling Grids for E-sciencE University of Coimbra GSAF Grid Storage Access Framework Salvatore Scifo INFN of Catania EGEE.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Data Management cluster summary David Smith JRA1 All Hands meeting, Catania, 7 March.
INFSO-RI Enabling Grids for E-sciencE Grid based telemedicine application for GATE Monte Carlo dosimetric studies using HOPE (Hospital Platform.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Data management in EGEE.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Architecture of LHC File Catalog Valeria Ardizzone INFN Catania – EGEE-II NA3/NA4.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Algiers, EUMED/Epikh Application Porting Tutorial, 2010/07/04.
NA-MIC National Alliance for Medical Image Computing Core 1b – Engineering Data Management Daniel Marcus Washington University.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Authentication, Authorisation and Security Emidio Giorgio INFN Catania.
2 nd EGEE/OSG Workshop Data Management in Production Grids 2 nd of series of EGEE/OSG workshops – 1 st on security at HPDC 2006 (Paris) Goal: open discussion.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Direct User Support and updates on EGEE.
Enabling Grids for E-sciencE EGEE-II INFSO-RI The Development of SRM interface for SRB Fu-Ming Tsai Academia Sinica Grid Computing.
INFSO-RI Enabling Grids for E-sciencE Security needs in the Medical Data Manager EGEE MWSG, March 7-8 th, 2006 Ákos Frohner on behalf.
EGEE Data Management Services
Federating Data in the ALICE Experiment
Jean-Philippe Baud, IT-GD, CERN November 2007
gLite Basic APIs Christos Filippidis
NA4/medical imaging. Medical Data Manager Installation
AMGA Metadata Service Vladimir Dimitrov IPP-BAS “gLite middleware Application Developers Course”, Sofia, Bulgaria,
Security and Replication of Metadata with AMGA
Cross-health enterprises Medical Data Management on the EGEE grid
Medical Data Manager use case: 3D medical images analysis workflow.
Introduction to Data Management in EGI
Encrypted Data Store, Hydra & Delegation Interface
Data Management cluster summary
AMGA Metadata Service Vladimir Dimitrov IPP-BAS “gLite middleware Application Developers Course”, Plovdiv, Bulgaria,
Data services in gLite “s” gLite and LCG.
The AMGA metadata catalog
Architecture of the gLite Data Management System
gLite Data and Metadata Management
INFNGRID Workshop – Bari, Italy, October 2004
Presentation transcript:

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 1 Dicom retrieval : overview of the DPM One command line to retrieve a file: lcg-cp The interface is a grid standard storage element interface. The DPM-DICOM library retrieve the file from a DICOM sever and make additional step before storing the file in a disk spool. DICOM server DPM Disks pool Standard interface Request a file Retrieve the file DPM- DICOM Plugin DPM- DICOM Library Copy the file

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 2 Dicom retrieval : get the dicom file DPM Request a file with the SFN DPM- DICOM Library DPM- DICOM Plugin The PFN associates with a DICOM file is handled by the DPM-DICOM plugin. The plugin make a DICOM transaction with the DICOM server to retrieve the medical picture. By default, the MDM uses the Conquest server as DICOM server, but any DICOM server can be used The database assocites each SFN with a PFN DICOM transaction with the DICOM protocol

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 3 Dicom retrieval : Anonymization and encryption DPM Disks pool Standard interface Retrieve the file DPM- DICOM Library DPM- DICOM Plugin Copy the final file DICOM file 1A1A 1B1B 2 Anonymous medical picture Step 1A: The DPM-DICOM uses the DCMTK library to anonymise the DICOM file Or Step 1B: The creaLibs convert the DICOM file to an inrimage. Step 2: The DPM-DICOM call Hydra to encrypt the final file DPM-DICOM use the RFIO library to copy the file in a spool disk. The spool disk is only a buffer for the file.

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 4 Register a DICOM file The file must be recorded in the DICOM server: PUSH DICOM Medical Data Manager (AMGA, LFC, DPM, Hydra, DICOM server) –A DICOM transaction can initiate the registration: –All this step can be done by a command line: –The file must be registered in all the components: AMGA LFC HydraDPM

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 5 Register in Hydra The first registration step is a security step The DICOM picture contains a unique identifier the SOP identifier. The Study/Serie identifier is unique for each study/serie. The hydra servers generate a key for the selected cypher. The cyper and the key is associated to the unique DICOM numbers analyze Study ID Serie ID Sop ID Select a cypher and generate a key DICOM image Hydra server s

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 6 Register in the DICOM server In the next step, the DPM-DICOM library records the DICOM picture in the DICOM server. The DPM-DICOM simulate a user call. It obtain the size of the final file. The size depends of : Size of the encrypted anonymous file DPM- DICOM Library DICOM file 1A1A 1B1B 2 Anonymous medical picture DICOM server The cyper and the key The fields erased in the anonymous step The size of the original file The DICOM server

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 7 Register references to the file A reference to a file is recorded in the DPM, but no copy of the file in the DPM disk pool is needed Directories with the Study, Sery and SOP identifier are created in the LFC The data erases during the anonymization step are registered in the AMGA server. - the size of the file - SFN and PFN - host of the disk pool LFN and SFN - size of the file - The metadata of the DICOM file DPM LFC AMGA

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 8 Strong Security To allow one user to access a medical file and its metadata the owner of the file must set the right in all the component : Example : LFC DPM Hydra AMGA

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 9 Fine-grained security The user can set the permission a group or a VO The user can set the ACL for an individual user (based on the DN) Query> getattr /mdm/PATIENT/b67982a3b0e825af054741a58e750ca5cc6278e name >> Marc-Elian Begin Query> getattr /mdm/IMAGE/guid:0617a6bf-09ba-428c-b5f4-f674312c5aa0 nx ny SOPinsUID >> 256 >> The permission can be set for : All the anonymous DICOM picture of a study All the anonymous DICOM picture of a serie An individual anonymous DICOM picture Each element of the metadata of a file

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 10 Metadata: AMGA AMGA is the ARDA METADA CATALOGUE PROJECT – The purpose is to store the metadata of the patient and of the medical picture AMGA is a front-end for PostgreSQL, MySQL, Oracle and SQLite database. The user VOMS credential can be used for the authentification The protocol is a streamed ASCII protocol with SSL encryption

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 11 Distributed metadata AMGA provides a powerfull but limited distributed schema : –Replication in AMGA follows an asynchronous, master-slave model, and supports partial replication of the directory hierarchy. –Slaves can replicate any sub-tree of the metadata hierarchy The MDM needs a full autonomy of the site –The metadata are stored and managed locally The MDM provide a library and a client that provide multi-site communication. This work is based on the AMGA client and used the same syntax. - Users can send the commands to all the servers or the selected one. - Users can dynamically add or remove servers AMGA

Enabling Grids for E-sciencE EGEE-II INFSO-RI Medical Data Manager 12 Distributed Key: Hydra Hydra have been developped at CERN by Akos Frohner It store key and cypher to encrypt/decrypt file. Each pair (key,cypher) is associated to a string The command line is: glite-eds-encrypt It is based on the Shamir's Secret Sharing algorithm –By default the MDM used 3 servers and 2 are need to decypher –Each site could install multiple hydra servers Hydra allow a strong security and very reliable service Part of the Key Part of the Key Part of the Key Key