1. Object Tracking Based on Appearance and Depth Information

2. outline
NN tracker
How to make use of top-view information

3. Challenges
changing appearance patterns of both the object and the scene;
object-to-object and object-to-scene occlusions;
camera motion
loss of information from 3D world on a 2D image,
—scene illumination changes
—complex object motion,
—real-time processing requirements. 3

4. NN Tracker
A tracking-by-detection framework , combines nearest-neighbor classification of bags of features
A framework that handles occlusion, background clutter, scale and appearance change
Steve Gu, Ying Zheng, and Carlo Tomasi, "Efficient Visual Object Tracking with Online Nearest Neighbor Classifier". (ACCV2010), Queenstown, New Zealand, November 8-12, 2010

5. -----The main advantages of tracking by detection come from
the flexibility and adaptability of its underlying representation of appearance.

6. Tracking-by-detection framework

8. Tracking result with dense-sift

9. Information from Top-view
Position
Distance

10. Initialize output : Input : obj & bg Model RGB Image
target boundingbox
pointsCloud
para of motion
output :
obj & bg Model
TVI
Convexhull
TransMatrix
Position/Scale
prediction

11. From RGB to TV

12. Predict the position in TVI
Given: position in previous TVI
Use transM
Nearest neighbor strategy

14. TVI to RGB
See RES1
Why don’t we just use this prediction to find obj ? It can’t recover from the fault of robot motion

15. Tracking Appearance model dense-sift feature
pathchsize :32, gridsize:8
Motion model
predicted position/ expected window size
See RES2

16. How to handle occlusion
Small occlusion:
threshold for matched feature
large occlusion
if report ‘losing target’, then discard current frame, keep current state, take another image….

17. How to handle object move or false position prediction
Means we can’t tracking the object in predict location.
Then search the whole image
Still can’t find matched object? Then
- stop .or.
- rotate to find object, if founded, take it as the first frame and proceeding further motion.

18. Main Challenge How many matched features means target is found?
Too large : miss the target
Too small : false positive
From testing : 20 now

19. Next step to try…. Only evaluate these convexhull
Possibility from position prediction