1. Dynamic Texture Mapping of 3D models for Stiffness map Visualization
Kartik Patath, R. Arun Srivatsan, Nico Zevallos, Howie Choset
Biorobotics Laboratory
This project is supported by NRI large grant IIS
INTRODUCTION
RESULTS
Tumor is a swelling of a part of the body, generally without inflammation, caused by an abnormal growth of tissue. They can be either benign or malignant (cancer causing).
Visualization in ROS visualizer (Rviz)
Figure 4(b):
Stiffness maps at different stages obtained during palpation of a silicone phantom tissue with embedded stiff inclusions.
Figure 4(a):
3D model of a liver in Rviz.
Tumors are stiffer than the surrounding tissues.
Forceful palpation can help in estimating the location, shape and stiffness of the tumor.
da Vinci needle drivel tool
Figure 4(c): Stiffness maps texture wrapped on to the model of the liver.
Augmented view of stiffness maps in a stereo viewer
Miniature force sensor
Phantom organ
Figure 5(b):
Model wrapped with a stiffness map
(shown in Rviz).
Figure 5(a):
3D model of a representative anatomy
Figure 1: Robotic palpation setup of Li et al. [1]
RELATED WORK
Several attempts have been made to visualize the stiffness data in an intuitive manner
Some of them use 2D/3D plots and some use stereo imagery [2].
Figure 5(c): Registered stereo point cloud with geometric model and overlaid stiffness map
Figure 5(d): Stereo overlay with transparent model
(a)
(b)
FUTURE WORK
REFERENCES
(c)
Augmenting a minimally invasive surgery has the potential to provide the surgeon better insight of the surgical process.
[1] L. Li, B. Yu, C. Yang, P. Vagdargi, R. A. Srivatsan, and H. Choset, “Development of an inexpensive tri-axial force sensor for minimally invasive surgery,” in Intelligent Robots and Systems. IEEE, 2017.
[2] T. Yamamoto, B. Vagvolgyi, K. Balaji, L. L. Whitcomb and A. M. Okamura, "Tissue property estimation and graphical display for teleoperated robot-assisted surgery," ICRA, 2009 
[3] R. A. Srivatsan, E. Ayvali, L. Wang, R. Roy, N. Simaan and H. Choset, “Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments”, ICRA 2016
[4] A. Talasaz and R. V. Patel, “Telerobotic Palpation for Tumor Localization with Depth Estimation”, IROS 2013.
Figure 2: Various approaches for representing stiffness data. (a) Srivatsan et al. [3] (b) Yamamoto et al. [2] (c) Talasaz et al. [4]
Figure 6(a): da Vinci robot, which has the ability to display augmented scenes of the surgery.
APPROACH
Sequence of steps used in texture mapping an image to a geometric model
Figure 6(b): Surgeon using Virtual reality set while observing the surgery.
Figure 3: Image courtesy of