1. SNM 2008 - Info SNMWhat IHE Delivers 1 Integrating the Healthcare Enterprise Image Fusion Todd Kantchev TK Medical Systems Jerold Wallis Washington University

2. SNM 2008 – InfoSNM 2 Image Fusion Scenario:  I want to view two studies (e.g. a PET and a CT) overlayed on the screen and  I want to look at it again later, maybe even on another screen

3. SNM 2008 – InfoSNM Your Problems with Image Fusion “I registered these before. Why do I have to do it again?”  Registrations not transferable between systems “Why can’t I see it on another workstation?”  Presentations not transferable between systems “Why can I only do PET / CT?”  No general purpose registration Impact  time lost to repeated work; delayed care  inability to access needed views 3

4. SNM 2008 – InfoSNM 4 How should it work? I scan on a dual system or several systems Some datasets are pre-registered The tech does additional registrations or tweaks the pre-registered sets On my workstation, I adjust the blending and read the images Later, on another workstation, I retrieve & view  perhaps tweak the registration  perhaps tweak the blending  perhaps register additional datasets

5. SNM 2008 – InfoSNM 5 IHE Solution (FUS) – Image Fusion  Record Registration between any datasets  Record Blending / Presentation parameters  Retrieve & View on any compliant workstation

6. SNM 2008 – InfoSNM Spatial Registration Object (Not required for already-registered PET-CT data from a hybrid scanner ) SRO Image 1 Image 2 Transformation 6

7. SNM 2008 – InfoSNM 7 Spatial Registration Object Encodes transformation to align images in a common registered space Identifies the registered space References the images to be aligned Not required for already-registered PET- CT data from a hybrid scanner

8. SNM 2008 – InfoSNM Blending Presentation State Image 1 Image 2 SRO Transformation BPS Window – 1 Window – 2 Color Table Transparency 8

9. SNM 2008 – InfoSNM 9 Blending Presentation Object References one set of underlying images (e.g. CT) References one set of superimposed images (e.g. PET) References (optionally) a Registration Object to align both image sets before blending. Encodes the window level for each set Encodes the blending factor (0 to 100%) of the overlay transparency

10. SNM 2008 – InfoSNM Essential Clinical Features MPR Fusion Side-by-side  Registered 10

11. SNM 2008 – InfoSNM 5/1/ 2007 SNM 2007 11 Blending state can be searched and retrieved from PACS  Can be used to retrieve the image data and the alignment automatically. User selects a Blending state.  Automatically load referenced objects. Alignment applied, PET re-sampled to CT and displayed fused automatically. Display is required to provide  Scrolling through volume  Adjustment of fusion blending and window levels  MPR  Side-by-side display Fusion- DisplayCT PET Registration Object Blending State

12. SNM 2008 – InfoSNM 12 Image Fusion Actors and Transactions Acquisition Modality Evidence Creator Image Manager ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Archive [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ Image Display Vol 1-20.1 Actors and Transactions in Image Fusion Profile

13. SNM 2008 – InfoSNM 13 Image Fusion Actors and Transactions Acquisition Modality Evidence Creator Image Manager ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Archive [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ Image Display

14. SNM 2008 – InfoSNM 14 Load CT and PET images Align images (automatically or manually) and fuse with desired blending factor Store the alignment, including  Registration object  Blending State (defining the blending factor, window levels and referencing all other objects) Storage can be on PACS or disk. Images do not need to be stored if already stored. Fusion- The Creating ProcessCT PET Blending State Registration Object

15. SNM 2008 – InfoSNM 15 Acquisition Modalities Hybrid Modalities are required to store Blending Softcopy Presentation State for the datasets, which are in the same Frame of Reference. Typically, the Referenced Spatial Registration Sequence is absent since no transformation needs to be applied to register the referenced images before display. All modalities on the IHE TF are now explicitly required, when creating image objects, not to use the same Frame of Reference for multiple image datasets unless all the images are in the same Reference Coordinate System. 4.57.4.1.2 Message Semantics – BSPS Stored

16. SNM 2008 – InfoSNM TEST FUS_MOD_STORE: Modality Stores Images and BSPS to Central Archive Special Instructions Acquistion Modalities should complete this test on the first day of the Connectathon to enable Evidence Creators, Image Display and Image Archives to complete their tests that depend on this one. No formal evaluation ? Validate BSPS? 16

17. SNM 2008 – InfoSNM 17 Image Fusion Actors and Transactions Acquisition Modality Evidence Creator Image Manager ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Archive [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ Image Display

18. SNM 2008 – InfoSNM 18 Evidence Creators Create and store Blending Softcopy Presentation objects as required  Referenced Spatial Registration Sequence (0070, 0404) references the Spatial Registration object, unless spatial registration is not required  The Presentation State Relationship Macro in the Blending Sequence (0070, 0402) lists all the images or frames to which blending is to be applied  Encodes presentation (blending, hue, windowing etc.) Create and store Spatial Registration Objects as required  Contains one or more Registration Sequences, each defining the transformation of the corresponding Original Dataset into the Registered Space.  Stored in the most recent of the Studies containing the registered images and in a different series from images and presentation states. Optionally create derived (re-sampled) images. 4.56.4.1.2 Message Semantics – Spatial Registration Stored 4.57.4.1.2 Message Semantics – BSPS Stored See also 4.18.4.1.2.5 for storage of Derived Images

19. SNM 2008 – InfoSNM Test: FUS_MOD_EC FUS_MOD_EC: Evidence Creator performs registration and blending on Modality image series Images will be on the Connectathon CENTRAL_ARCHIVE after Modalities complete their FUS_MOD_STORE test. This is a test between Modality and Evidence Creator actors 19

20. SNM 2008 – InfoSNM Demonstrate that Evid. Creator …register image datasets which do not share a Frame of Reference, using translations, rotation & scaling, as required …(recommended) that it notes the method of registration within the Spatial Registration object. …it can set the contrast/window settings for the underlying grayscale dataset, and … it can set the windowing and color map for the overlay dataset … it can set the transparency for the overlay dataset 20

21. SNM 2008 – InfoSNM FUS_MOD_EC Create several examples of the presentation state, with different registration and window/blending settings (eg, different transparency or color maps) Store them to the archive for use by Displays. 21

22. SNM 2008 – InfoSNM 22 Image Fusion Actors and Transactions Acquisition Modality Evidence Creator Image Manager ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Archive [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ Image Display

23. SNM 2008 – InfoSNM 23 Image Manager/Archive Able to store, query and retrieve Blending Softcopy Presentation States, supporting all required keys, already defined in the TF. Note that SOP Class UID (a required matching key for the SCP) is a useful way to limit responses to queries for Blending Softcopy PS. Able to retrieve Spatial Registration Objects. Retrieval of images must be supported at Series Level. 4.15.4.1.2 Message Semantics- Query Softcopy PS

24. SNM 2008 – InfoSNM 24 Image Fusion Actors and Transactions Acquisition Modality Evidence Creator Image Manager ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations Image Archive [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ Image Display

25. SNM 2008 – InfoSNM 25 Image Display Able to retrieve, read and apply to images Blending Softcopy Presentation States and, if referenced, Spatial Registration Objects as required Able to retrieve the referenced images, if not already present locally, and perform image registration as required if necessary Display images, supporting specific display features (fusion, side by side, MPR, adjustment of blending, windowing- aka NM, navigation through spatial and time dimensions, display content of specific attributes etc.) 4.17.4.2.3 Retrieve/View Presentation States. 4.58.4.1.2 Retrieve Spatial Registrations

26. SNM 2008 – InfoSNM Essential Clinical Features MPR Fusion Side-by-side  Registered 26

27. SNM 2008 – InfoSNM 27 50% Blending Presentation Overlay Transparency 20% 50% 100% 80% 50%

28. SNM 2008 – InfoSNM 5/1/ 2007 SNM 2007 28 Blending state can be searched and retrieved from PACS  Can be used to retrieve the image data and the alignment automatically. User selects a Blending state.  Automatically load referenced objects. Alignment applied, PET re-sampled to CT and displayed fused automatically. Display is required to provide  Scrolling through volume  Adjustment of fusion blending and window levels  MPR  Side-by-side display Fusion- DisplayCT PET Registration Object Blending State

29. SNM 2008 – InfoSNM Example- Side by Side Side-by-side can be up/down (PET/CT) Side-by-side can omit fusion image when displayed in this format Recommended – MIP image capability (upper left) 29

30. SNM 2008 – InfoSNM Example- Side by Side Side-by-side can be up/down, can be multiple levels (eg 3 levels as shown here) Side-by-side can include fusion image (preferred) 30

31. SNM 2008 – InfoSNM Example- Side by Side Side-by-side can include multiple CT window settings (as shown here) Side-by-side can include fusion image (preferred) 31

32. SNM 2008 – InfoSNM Example- Side by Side PET-only image (upper right) can be pure BW (preferred as on prior slide, but not mentioned in profile), but can use same LUT as fused image as shown here 32

33. SNM 2008 – InfoSNM Example- MPR MPR screen must be able to show 3 planes MPR screen need not show side-by-side as well, but can do so (here with pure CT image at upper left, and fused below) 33

34. SNM 2008 – InfoSNM Example- MPR Must be able to adjust blending (transparency) Same screen adjusted to be mostly CT 34

35. SNM 2008 – InfoSNM Example- MPR Must be able to adjust blending (transparency) Same screen adjusted to be mostly PET 35

36. SNM 2008 – InfoSNM Example- MPR Different vendor example for MPR fused 36

37. SNM 2008 – InfoSNM Fusion controls Must have controls for  Windowing CT (using standard center/width controls)  Windowing PET/NM (using separate upper & lower level controls, not center/width controls  Adjusting transparency  Navigating volume in three planes  Changing color LUT table for overlay is recommended but not required 37

38. SNM 2008 – InfoSNM Example- Controls 38 Must have controls for  Windowing CT (using standard center/width controls)  Windowing PET/NM (using separate upper & lower level controls, not center/width controls  Adjusting transparency  Navigating volume in three planes  Changing color LUT table for overlay is recommended but not required

39. SNM 2008 – InfoSNM Example- Controls 39 Must have controls for  Windowing CT (using standard center/width controls)  Windowing PET/NM (using separate upper & lower level controls, not center/width controls  Adjusting transparency  Navigating volume in three planes  Changing color LUT table for overlay is recommended but not required

40. SNM 2008 – InfoSNM Example- Controls 40 Must have controls for  Windowing CT (using standard center/width controls)  Windowing PET/NM (using separate upper & lower level controls, not center/width controls  Adjusting transparency  Navigating volume in three planes  Changing color LUT table for overlay is recommended but not required

41. SNM 2008 – InfoSNM Selecting Studies Users must be able to select the softcopy presentation state (BSPS) More than one BSPS may be present If not already present, any images or spatial registration objects referenced in the Presentation State shall be retrieved by the Image Display. (For clarification on this, see closed issue 19 and section 4.17.4.2.3 ) 41

42. SNM 2008 – InfoSNM Full Profile on IHE Site http://www.ihe.net/Technical_Framework/ index.cfm#radiology http://www.ihe.net/Technical_Framework/ index.cfm#radiology Open link on page above for FUS profile 42