1. An Effective & Interactive Approach to Particle Tracking for DNA Melting Curve Analysis 李穎忠 DEPARTMENT OF COMPUTER SCIENCE & INFORMATION ENGINEERING NATIONAL TAIWAN UNIVERSITY

2. DNA Melting Curve Analysis  Used for the detection of DNA sequence variants  DNA Melting Analysis in Temperature-Gradient Micro-channel Temperature-Gradient Micro-channel Heater Carrier (Bead/Droplet) Thermometer Substrate 1/54

3. DNA Melting Curve Analysis Temperature Fluorescent Intensity Melting Temperature 2/54

4. DNA Melting Curve Analysis 3/54

5. Motivation  People label each particles (carrier) frame by frame  That is time-consuming  We design an annotation tool to reduce human effort 4/54

6. Related Work  Particle tracking ParticleTracker: An ImageJ plugin for multiple particle detection and tracking [Sbalzarini et al., Journal of structural biology 2005] u-track [Jaqaman et al., Nature Methods 2008]  Interactive video annotation Tracking with active learning [Vondrick et al., NIPS 2011] Interactive object detection [Yao et al., CVPR 2012] 5/54

7. Proposed System User annotation Detection of bounding circle of the particle Acquisition of labels at other frames by tracking the particle User correction Update of tracker & labels Acquisition of all correct labels 6/54

8. Detecting Bounding Circle of a Particle Median filter Otsu's method Edge detection Least-squares fitting Dilation Erosion 7/54

9. Least-Squares Fitting of Bounding Circle 8/54

10. Least-Squares Fitting of Bounding Circle 9/54

11. Possible Choices of Trackers  Linear interpolation  Correlation filter based tracker [Zhang et al., ECCV 2014]  Normalized cross-correlation matching 10/54

12. Linear Interpolation 1 2 3 4 5 6 7 8910 11 12 11/54

13. Linear Interpolation: User Correction 1 2 3 4 5 6 7 8910 11 12 12/54

14. Linear Interpolation: Update of Labels 1 4 5 6 7 8910 11 12 2 3 13/54

15. Linear Interpolation: Update of Labels 1 4 5 6 7 8 9 10 11 12 2 3 14/54

16. Linear Interpolation: User Correction 1 2 3 4 5 6 7 8 9 10 11 12 15/54

17. Correlation Filter Based Tracker [Zhang et al., ECCV 2014] 16/54

18. Online Update of Filter Frame 1 17/54

19. Online Update of Filter Frame 2 18/54

20. 1 2 One-Way Method 19/54

21. 1 2 One-Way Method 20/54

22. 1 2 One-Way Method 3 21/54

23. 1 2 One-Way Method 3 22/54

24. 1 2 345 67 8 9 10 11 12 13 14 Two-Way Method 23/54

25. 1 2 35 67 8 9 10 11 12 13 14 Two-Way Method 4 24/54

26. 1 5 67 8 9 10 11 12 13 14 Two-Way Method 4 2 3 25/54

27. 1 5 67 8 9 10 11 12 13 14 Two-Way Method 4 2 3 26/54

28. 1 7 8 9 14 Two-Way Method 4 2 3 5 6 10 11 12 13 27/54

29. 1 7 8 9 1011 12 13 14 Two-Way Method 4 2 3 5 6 28/54

30. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Two-Way Method 29/54

31. Normalized Cross-Correlation Matching  Given a image f and template t, normalized cross-correlation (NCC) measures the similarity between each part of f and t: TemplateInput imageOutput NCC 30/54

32. Normalized Cross-Correlation Matching Template Frame 1 31/54

33. Normalized Cross-Correlation Matching Frame 2 32/54

34. 1 2 One-Way Method 33/54

35. 1 2 One-Way Method 34/54

36. 1 2 One-Way Method 3 35/54

37. 1 2 One-Way Method 3 Update the template 36/54

38. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Two-Way Method 37/54

39. Failure in Tracking with Normalized Cross-Correlation Template of particle 1 1 2 38/54

40. Combining NCC & Extrapolation Frame t-2Frame t-1Frame t 1 2 1 2 1 2 x x x 39/54

41. Combining NCC & Extrapolation NCCScore of predicted location Combined score 40/54

42. Experiments  Evaluate how much human effort our system can reduce  Simulate the process of annotating video with our system  Evaluation metric Number of manual annotation  Count a tracked bounding box as a correct label if the distance between the centers of it and the ground-truth bounding box is not more than 10 pixels 41/54

43. Methods  Interp  CF-1way  CF-2way  NCC-1way  NCC-2way  NCC-Extrap-1way  NCC-Extrap-2way 42/54

44. The Order of Labeling  For those methods not restricting the order of labeling Always correct the label with maximum center location error  For other methods Same as the video display order 43/54

45. Video Dataset Name# frames# particles# annotations Droplet1120315635 Droplet2637534192 Bead4205727  Video Droplet 1 is for parameter tuning which is performed using brutal force search 44/54

46. Parameter Tuning for CF- 1way 45/54

47. Parameter Tuning for CF- 1way 46/54

48. Parameter Tuning for NCC-Extrap-1way 47/54

49. Parameter Tuning for NCC-Extrap-1way 48/54

50. Result Droplet2 (# annotations = 4192) Bead (# annotations = 727) Interp457 (10.90%)88 (12.10%) CF-1way1475 (35.19%)79 (10.89%) CF-2way1973 (47.07%)112 (15.41%) NCC-1way56 (1.34%)11 (1.51%) NCC-2way129 (3.08%)21 (2.89%) NCC-Extrap-1way53 (1.26%)9 (1.24%) NCC-Extrap-2way115 (2.74%)20 (2.75%) 49/54

51. Error Analysis for NCC-Extrap-1way 50/54

52. Error Analysis for NCC-Extrap-1way 51/54

53. Error Analysis for NCC-Extrap-1way 52/54 Target Error

54. Conclusions  We designed a system for particle annotation in video sequences  Our system can reduce human effort in annotation  Combining NCC and extrapolation achieves the best result  It is better to annotate video in its display order  Future work Use polynomial curve fitting to predict the location of particle in the next frame 53/54

55. Thank you for listening