1. The past and future of virtual reality simulation in neurologic surgery Longwei F. 2014.9.26

2. Contents 1.The development of surgery simulation 2.Technical in a virtual reality simulator 3.Simulation devices 4.Example of virtual reality surgery 5.Future of virtual reality surgery simulation

3. The development of surgery simulation 1890, Willam Halsted, apprenticeship-based model 1910, Abraham Flexner, Flexner Report, scientific foundation of medicine gained recognition computer generated graphics, entire human anatomy in virtual space

4. The development of surgery simulation 300 AD, Galen, human anatomic description based on animals 15 th century, Leonardo, accurate anatomic drawings and rendering 1543, Vesalius, humani corporis fabrica 18 th century, Ercole Lelli, anatomic human waxworks 1960, Asmund Laerdal, resusci-Ann mannequin 1980, computer generated graphics, entire human anatomy in virtual space 1994, national library of medicine, visible human project Immersive virtual reality

5. Technical in a virtual simulator Data( MRI, CT, PET, Ultrasound ) Image segmentation Regional image segmentation, marginal image segmentation, neural network, fuzzy clustering algorithm segmentation Graphic rendering( Surface rendering, volume rendering) Graphic display( Holographic or 3D device) Physical modeling (Mass-spring, FEM) Collision detection (Spatial partitioning, bounding volume hierarchies) Haptic interaction Articulating arms, tactile sensor

6. Simulation devices The original simulator, for catheter insertion, no feedback Virtual brain project, for ventriculostomy surgery, have feedback

7. Simulation devices Dextroscope(VizDexter) (3d modal, position controllers, Holographic image) (coregister data, perform segmentation, make measurements, simulate multiple different intraoperative viewpoints) Bone removal(lock of haptic feedback) Intracranial aneurysm clipping Temporal bone Cranial base surgery

8. Simulation devices Cranial base surgery simulations Interactive virtual dissector Data(stereoscopic photographic from cadaveric dissections) Haptic feedback Dissection field is limited with fixed view No deform TempoSurg Display same as Dextroscope, Haptic feedback using PHAMToM arm Otolaryngology Temporal and petrous bone,

9. Simulation devices ROBO-SIM Produce real time deformation of virtual tissue via mass spring models NEUROBOT, a minimally invasive robotic arm Haptic feedback Mainly for pre-operative Small anatomic structures(<1mm) are not adequately visualized Limited by the absence of membranes and blood

10. Simulation devices ImmersiveTouch Volume rendering models Use electromagnetic head-tracking goggles to register user position, the perspective adjust to user movement Can analyze the location of the catheter tip Haptic feedback with user perspective Ventriculostomy, occipital trajectories, ventriculostomy on slit ventricles

11. Example of virtual reality surgery Principle: An AV scene includes a 3-D surface mesh of the operative field reconstructed from pair of stereoscopic images acquired through surgical microscope, and 3-D surfaces segmented from preoperative multimodal images of the patient.

12. Example of virtual reality surgery Methods : 1.Build preoperative3-D model. MEG and fMRI datasets were registered on MRI, whose coordinate system is image coordinate system. 2.Define the coordinate system of patient’s head as physics coordinate system 3.Change the 3-D model to the physics coordinate. The transformation is.

13. Example of virtual reality surgery Methods : 4.Build the AR image. And change the image to the physics coordinate. The transformation is as follows:

14. Example of virtual reality surgery Methods : 5.The AV and AR both expressed in the physical coordinate system. Merging reality and virtuality did not require and addition registration. 6.The neurosurgeon was able to visualize this 3-D scene and interact with it by toggling each component of the scene on and off.

15. Example of virtual reality surgery Case: Patient: a left frontal low-grade glioma on a 31-year-0ld male An initial surface mesh of the patient skin was reconstructed. Before the scalp incision and elevation, the second reconstruction perform to help understand the spatial relationship.

16. Example of virtual reality surgery After opening the bone, stereoscopic image acquistion of the dura mater was performed. During dura mater opening, A surface mesh of the arachnoid was reconstructed.

17. Example of virtual reality surgery Conclusion: 1.Preoperative planning 2.Help the neurosurgeon understand the spatial relationship of the tissue in operation 3. The results obtained were promising: 3.05±1.54 mm with a maximum distance of 6 mm.

18. Future of virtual reality surgery 1.More complex surgery 2.Simulator can fulfill multi-task 3.Haptic feedback is more accurate 4.Computation and display in real time