Dynamic Texture Mapping of 3D models for Stiffness map Visualization Kartik Patath, R. Arun Srivatsan, Nico Zevallos, Howie Choset Biorobotics Laboratory This project is supported by NRI large grant IIS1426655 INTRODUCTION RESULTS Tumor is a swelling of a part of the body, generally without inflammation, caused by an abnormal growth of tissue. They can be either benign or malignant (cancer causing). Visualization in ROS visualizer (Rviz) Figure 4(b): Stiffness maps at different stages obtained during palpation of a silicone phantom tissue with embedded stiff inclusions. Figure 4(a): 3D model of a liver in Rviz. Tumors are stiffer than the surrounding tissues. Forceful palpation can help in estimating the location, shape and stiffness of the tumor. da Vinci needle drivel tool Figure 4(c): Stiffness maps texture wrapped on to the model of the liver. Augmented view of stiffness maps in a stereo viewer Miniature force sensor Phantom organ Figure 5(b): Model wrapped with a stiffness map (shown in Rviz). Figure 5(a): 3D model of a representative anatomy Figure 1: Robotic palpation setup of Li et al. [1] RELATED WORK Several attempts have been made to visualize the stiffness data in an intuitive manner Some of them use 2D/3D plots and some use stereo imagery [2]. Figure 5(c): Registered stereo point cloud with geometric model and overlaid stiffness map Figure 5(d): Stereo overlay with transparent model (a) (b) FUTURE WORK REFERENCES (c) Augmenting a minimally invasive surgery has the potential to provide the surgeon better insight of the surgical process. [1] L. Li, B. Yu, C. Yang, P. Vagdargi, R. A. Srivatsan, and H. Choset, “Development of an inexpensive tri-axial force sensor for minimally invasive surgery,” in Intelligent Robots and Systems. IEEE, 2017. [2] T. Yamamoto, B. Vagvolgyi, K. Balaji, L. L. Whitcomb and A. M. Okamura, "Tissue property estimation and graphical display for teleoperated robot-assisted surgery," ICRA, 2009  [3] R. A. Srivatsan, E. Ayvali, L. Wang, R. Roy, N. Simaan and H. Choset, “Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments”, ICRA 2016 [4] A. Talasaz and R. V. Patel, “Telerobotic Palpation for Tumor Localization with Depth Estimation”, IROS 2013. Figure 2: Various approaches for representing stiffness data. (a) Srivatsan et al. [3] (b) Yamamoto et al. [2] (c) Talasaz et al. [4] Figure 6(a): da Vinci robot, which has the ability to display augmented scenes of the surgery. APPROACH Sequence of steps used in texture mapping an image to a geometric model Figure 6(b): Surgeon using Virtual reality set while observing the surgery. Figure 3: Image courtesy of http://web.inc.bme.hu/csonka/csg/oktat/www/wrl/texture/start.html