1. 3D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay
Hongen Liao, Takashi Inomata, Ichiro Sakuma and Takeyoshi Dohi
Presented by Zhenzhou Shao
2/24/2011

2. Outline Introduction Material and Methods Experiment and Results
System Configuration
IV Image Display and Overlay Device
Registration of Spatial 3D Image in Patient
Software Alignment
Surgical Procedure
Experiment and Results
Conclusion

3. Introduction Magnetic resonance imaging (MRI)
A medical imaging technique
Provides detailed information about soft tissue.
The obtained images are used to accurately identify treatment areas by acquiring pre-/intraoperative information and updating it to a navigation system used in MRI-guided surgery.
A strong magnetic field is created.
protons in the body to align themselves.
radio waves are absorbed by the protons with the same frequency, and protons will be raised to a higher state of energy.
When the radio signal is turned off, after a period of time, the protons will return to their original energy state. The extra energy is received by the sensors in the scanner, then sent to a computer which processes all the signals and generates it into an image

4. Introduction Potential efficacy using MRI-guided surgery Advantages
Enhance the surgeon’s capability
Decrease the invasiveness of surgical procedure
Increase the accuracy and safety
Disadvantages
Display of a set of 2D sectional images
Hand–eye coordination problem
Repetitive look away from the patient
The surgeon had to reconstruct 3-D information in their mind. However, the reconstructed information sometimes differs between individual surgeons.

5. Introduction Augmented Reality (AR) Superimpose the virtual model
into the real scene.
Video see-through AR
Head mounted display (HMD)
Limited field of view
A lag for motion parallax
Cannot provide a natural view
for multiple observers

6. Introduction Optical see-through AR
Using a semi-transparent mirror for merging virtual model with a direct view.
Surgeon can see through the body.
Enhance the surgeon’s ability to
perform a complex procedure.
Depth information is required.

7. Introduction Integral Videography(IV)
Figure 1 shows the principle of IV. Each point in a 3-D space is reconstructed by the convergence of rays from pixels on the computer
display through lenses in the array. The observer can see any point on the display from various directions as if it were fixed in 3-D space. IV can display animated 3-D objects, and it also has the merits of IP.

8. System Configuration

9. IV Image and Overlay Device

10. Registration of 3D Image in Patient

11. Registration of 3D Image in Patient
Project an IV image of a spatial calibration model with special calibration points and reflect the image into 3-D space via the half-slivered mirror

12. Registration of 3D Image in Patient

13. Registration of 3D Image in Patient

14. Software Alignment

15. Surgical Procedure Calibrate the position of reflected IV image;
Place sterile fiducial markers on the surface of the patient’s body and scan the target area;
Segment the target of interest and markers from the MRI data. Perform patient-to-image registration to find the ;

16. Surgical Procedure
Render the IV images and transfer them to the overlay device;
Perform the surgical treatment under the guidance of IV image overlay;
After finishing the treatment, translate the patient into the scanner again and confirm surgical result.

17. Experiment and Results
Accuracy measurement
Implemented by using markers in a phantom simulating the human head.

18. Accuracy measurement
Five markers for registration and two for error measurement.
Marker: 10 mm in external diameter and 3 mm in internal diameter.
The distance between the center of the actual marker and that of the spatial projected IV marker was measured as an overlay error.
The mean value of the error was 0.90 mm, and the standard deviation was 0.21 mm

19. Targeting Experiment
Compare the procedure time and success rate of targeting an object using 2-D image guidance and IV overlay system guidance.
Phantom consisted of a plastic
cube container filled with
an agar.

20. Targeting Experiment Six MRI markers were attached.
Three sets of acrylic cylinders
with diameters of 1.5, 2 and
3 mm were embedded
within the phantom.

21. 2-D image guidance
four operators for each of ten trials

22. IV overlay system guidance

23. Results of guidance
2-D image guidance
IV overlay
system guidance

24. Comparison of procedure time

25. Feasibility Evaluation
Evaluate the feasibility by a volunteer test.
Scan brain using MRI.
Motion parallax could be generated due to the motion of an observer.
The motion parallax of IV autostereoscopic brain images combined with the volunteer’s head was taken from various directions.

26. Feasibility Evaluation

27. In Vivo Animal Experiment
Target a pig’s gallbladder.
A set of markers was attached to the skin of the surgical area.

28. In Vivo Animal Experiment
Surgical planning to minimizing the surgical exposure.
Surgical instrument is tracked.
The targeting experiment was performed by a medical doctor.
The result shows that the developed intraoperative IV image overlay technique with corresponding image registration can
improve surgical navigation by providing a direct and intuitive view of the operation field. In combination with robotics, it can
even supply guidance by predefining the path of a needle or by preventing the surgical instruments from moving into critical
regions.

29. Conclusion
An autostereoscopic image overlay system for MRI-guided surgery is developed.
IV is employed to provide accurate 3-D spatial images and reproduces motion.
A fast and accurate spatial image registration method was developed.
Safe, easy, and accurate surgical diagnosis and therapy.

30. Questions?