1. C2A: Crowd Consensus Analytics for Virtual Colonoscopy
Ji Hwan Park, Saad Nadeem, Seyedkoosha Mirhosseini, and Arie Kaufman

2. Virtual Colonoscopy (VC)
3D colon reconstruction from a CT scan
Navigation inside a colon to detect a polyp
Interpretation time: avg. 30min

3. Crowdsourcing VC Crowd aided detection tool
Create VC videos (5 min) - Polyp size: >10mm
Divide into short video segments
Two 10 second training videos
Mark 80% of the video segments as polyp-free
Cecum
Rectum

4. Contributions A medical crowdsourcing visual analytics platform
Find optimal crowd and application specific parameters
Detect user and video segments anomalies
Build a crowd consensus on polyp and polyp-free video segments

5. Interactive Visual Analytics
Workflow
Statistical Analysis
Input data
Crowd
Clinical Experts
Analysts
Existing approach
Input data
Crowd
Clinical Experts
Interactive Visual Analytics
Analysts
Our approach

6. Input data Viatronix V3D-Colon VC system for video segments
Total 8 datasets
4 datasets: 136 video segments with 120°FOV and speed
4 datasets: 163 video segments with 90° FOV and 50 speed

7. Input data
Data collection - Amazon Mechanical Turk

8. Task analysis T1: Effects of crowd and application specific parameters
Rewards, demographics
T2: Anomaly detection
Users, video segments
T3: Crowd consensus on polyp and polyp-free video segments

9. Overview Timeline Filtering View Aggregated Textual Information
Word Cloud
Aggregated Textual Information
Crowd View
Consensus Map
Similarity View

10. Consensus Map (T1,T2,T3)

11. Consensus Map (T1,T2,T3) Crowd view Consensus slider50
Consensus slider
Video segment
Direction of fly-through
Aggregated video
segment information
Demographics
Aggregated user information
User

12. Crowd view (T1)
Show the distributions and the relationships between parameters
Parallel set
Consensus map
Crowd view

13. Similarity View (T1,T2) Timing # of polyps/incorrect answers
Find optimal user performance and quality parameters
Detect anomaly video segments
MDS or t-SNE
Timing
# of polyps/incorrect answers
Users
Video segments

14. Timeline Filtering View
Date
# of datasets

15. Aggregated Textual Information (T3)
An average of information
Distributions for sensitivity (SE) and specificity (SP)

16. Word Cloud (T2)

17. Case Studies Case 1 - Higher FOV and Higher Speed Dataset
136 workers from USA, India
Case 2 - Lower FOV and Lower Speed Dataset
144 workers from USA

18. Case 1: Higher FOV and Higher Speed Dataset
Crowd SP: 85.2%, SE: 65.2% , Expert SP: 72.7%, SE: 92.9%

19. Case 2: Lower FOV and Lower Speed Dataset
Crowd SP: 80.0%, SE: 82.4%, Expert SP: 87.2%, SE: 86.7%

20. Case 2: Lower FOV and Lower Speed Dataset

21. Feedback from domain experts
Two VC radiologists
Overall positive feedback
Consensus view - clearly showed the polyp and polyp-free regions
Word cloud - showed the keywords from the user feedback on our VC video data.
Crowd view - a little confusing in the beginning

22. Conclusion Crowd consensus analytics platform for VC
Build a consensus on polyp and polyp-free video segments
Detect anomalous users and video segments
Find optimal crowd and application specific parameters
Help discard 80% of video segments
 Reduce the interpretation time and associated cost of radiologists

23. Acknowledgement: This research has been partially supported by the NSF grants
CNS ,IIP , and CNS
Q&A