1. Enhancing Fundamentals of Laparoscopic Surgery Trainer Box via Designing A Multi-Sensor Feedback System Qiongjie Tian, Lin Chen and Baoxin Li {Qiongjie.Tian, Lin.Chen.6, Baoxin.Li} @asu.edu Computer Science and Engineering, Arizona State University INTRODUCTION INTERFACE DESIGN AND FEEDBACK MECHANISM HARDWARE SETUP Laparoscopic surgery has become popular for its potential advantages of shorter hospital stay, lower risk of infection, and smaller incision, and so on [1]. However, it requires a surgeon to have good cognitive and motor skills [2]. The Fundamentals of Laparoscopic Surgery (FLS) Program [3] has been used in many hospitals/training centers across the country to improve minimally-invasive surgery education via simulation-based training. The basic FLS Trainer Box is mostly a passive platform that is unable to provide any online or offline feedback to a trainee. Senior surgeons may be invited to evaluate a trainee’s performance to provide feedbacks or instructions [4]. However, that would be costly and cannot be always available due to the limited human resources. We propose a design to expand the FLS box into an intelligent system that supports real-time and offline feedback based on automated analysis of a trainee’s performance. REFERENCES CONCLUSION We present a software system as an enhancing extension of FLS box for both online and off-line feedbacks. The hardware setups, interface design, and feedback mechanisms of the proposed system are described. The new system was found to be easy to use for anyone who is unfamiliar with the original FLS box. Live feedbacks: The PC in the system is employed to process the captured streams in real-time and some key skill-related attributes such as smoothness of motion and acceleration can be displayed on the side bar on the monitor[Fig. 4]. The system is configurable in that the trainee can choose to exclude any of the windows from being display so as to fit his/her preference and learning habit. The original FLS box [Fig.1] contains a box and a set of accessories to simulate the minimal- invasive surgery inside human body. A on- board camera will capture the operation inside the box and display on a monitor. The original FLS box has no sensor to capture user’s out-side-box operation and hand movement for future motion analysis and feedback. [1] M. Carbajo, J. Martin del Olmo, J. Blanco, C. De la Cuesta, M. Toledano, F. Martin, C. Vaquero and L. Inglada, "Laparoscopic treatment vs open surgery in the solution of major incisional and abdominal wall hernias with mesh," Surgical endoscopy, vol. 13, no. 3, pp. 250-252, 1999. [2] C. Reiley, H. Lin, D. Yuh and G. Hager, "Review of methods for objective surgical skill evaluation," Surgical endoscopy, vol. 25, no. 2, pp. 356-366, 2011. [3] "FLS trainer box manual," Available at http://www.flsprogram.org/wp-content/uploads/2012/ 07/Manual-Skills-Guidelines.pdf. [4] T. Grantcharov, L. Bardram, P. Funch-Jensen and J.Rosenberg, "Assessment of technical surgical skills," European Journal of Surgery, vol. 168, no. 3, pp. 139- 144, 2002 To capture the trainee’s hand movement there are two additional cameras beside the FLS box on-board video camera: a USB webcam and a Kinect camera. Data gloves with attached motion trackers may be added to collect real-time location data of trainees’ hands, which can be used to compute motion statistics, like speed. The entire system is connected with a PC which can be used to store and process captured video frames and motion data. Fig 1. This is the real system we designed: in the dash rectangle, it is the previous FLS box whose monitor only shows the tools’ movements; the other parts are added by us to enrich FLS box’s capabilities Fig 2. Register interface (left) and Login interface (right). Data archival, indexing and retrieval: Our system supports this functionality while the original FLS box is a “pass-through” system. [Fig. 2] Offline feedback: It is desirable to allow a trainee to compare his/her performance between different practice sessions as this would provide insights as to how to improve. This goes beyond simply providing videos from two sessions to the trainee, since computational tools can be used to deliver comparative attributes based on the videos. [Fig. 3] Fig 3. Illustrating retrieval and offline feedback: The left panel lists the previous trials retrieved for a given user. By selecting two videos, say the first and last ones, the graph window displays computed motion attributes for the two selected sessions, while the text panels below the graph supply other comparative results (computed by an underlying module). Fig 4. The display window in action: