C2A: Crowd Consensus Analytics for Virtual Colonoscopy Ji Hwan Park, Saad Nadeem, Seyedkoosha Mirhosseini, and Arie Kaufman

Virtual Colonoscopy (VC) 3D colon reconstruction from a CT scan Navigation inside a colon to detect a polyp Interpretation time: avg. 30min

Crowdsourcing VC Crowd aided detection tool Create VC videos (5 min) - Polyp size: >10mm Divide into short video segments Two 10 second training videos Mark 80% of the video segments as polyp-free Cecum Rectum

Contributions A medical crowdsourcing visual analytics platform Find optimal crowd and application specific parameters Detect user and video segments anomalies Build a crowd consensus on polyp and polyp-free video segments

Interactive Visual Analytics Workflow Statistical Analysis Input data Crowd Clinical Experts Analysts Existing approach Input data Crowd Clinical Experts Interactive Visual Analytics Analysts Our approach

Input data Viatronix V3D-Colon VC system for video segments Total 8 datasets 4 datasets: 136 video segments with 120°FOV and 100 speed 4 datasets: 163 video segments with 90° FOV and 50 speed

Input data Data collection - Amazon Mechanical Turk

Task analysis T1: Effects of crowd and application specific parameters Rewards, demographics T2: Anomaly detection Users, video segments T3: Crowd consensus on polyp and polyp-free video segments

Overview Timeline Filtering View Aggregated Textual Information Word Cloud Aggregated Textual Information Crowd View Consensus Map Similarity View

Consensus Map (T1,T2,T3)

Consensus Map (T1,T2,T3) Crowd view Consensus slider 50 Consensus slider Video segment Direction of fly-through Aggregated video segment information Demographics Aggregated user information User

Crowd view (T1) Show the distributions and the relationships between parameters Parallel set Consensus map Crowd view

Similarity View (T1,T2) Timing # of polyps/incorrect answers Find optimal user performance and quality parameters Detect anomaly video segments MDS or t-SNE Timing # of polyps/incorrect answers Users Video segments

Timeline Filtering View Date # of datasets

Aggregated Textual Information (T3) An average of information Distributions for sensitivity (SE) and specificity (SP)  

Word Cloud (T2)

Case Studies Case 1 - Higher FOV and Higher Speed Dataset 136 workers from USA, India Case 2 - Lower FOV and Lower Speed Dataset 144 workers from USA

Case 1: Higher FOV and Higher Speed Dataset Crowd SP: 85.2%, SE: 65.2% , Expert SP: 72.7%, SE: 92.9%

Case 2: Lower FOV and Lower Speed Dataset Crowd SP: 80.0%, SE: 82.4%, Expert SP: 87.2%, SE: 86.7%

Case 2: Lower FOV and Lower Speed Dataset

Feedback from domain experts Two VC radiologists Overall positive feedback Consensus view - clearly showed the polyp and polyp-free regions Word cloud - showed the keywords from the user feedback on our VC video data. Crowd view - a little confusing in the beginning

Conclusion Crowd consensus analytics platform for VC Build a consensus on polyp and polyp-free video segments Detect anomalous users and video segments Find optimal crowd and application specific parameters Help discard 80% of video segments  Reduce the interpretation time and associated cost of radiologists

Acknowledgement: This research has been partially supported by the NSF grants CNS0959979,IIP1069147, and CNS1302246. Q&A