1. Handover and Tracking in a Camera Network Presented by Dima Gershovich

2. Decentralized Discovery of Camera Network Topology Ryan Farrell, Larry Davis

3. Motivation - 1 ► Networks of tens and hundreds of cameras are installed for coverage of large areas ► For collaboration between cameras an automatic recovery of network topology is needed ► A particular interest is in problem where the cameras have a non-overlapping field of view (thus, finding links between cameras is not trivial because of the passage time gaps..)

4. Motivation - 2 ► Centralized solution is computationally expensive and scales poorly for large networks. ► Distributed solution provides a better scalability.

5. Notions -1 ► Network Topology  Two cameras are considered adjacent if there exists a path between the cameras that does not cross through fields of view of any other cameras  Graph that defines the network topology: ► Nodes – cameras ► Edges – between adjacent cameras

6. Notions - 2 ► Transition Model A probability distribution that describes:  Where objects go when they leave one camera  How much time it takes to arrive to the next camera ► Transition model implicitly defines the network topology ► Finding the transition model automatically is our goal

7. Notations ► C i – camera ► f – an appearance ► A(f) – the density of the appearance ► D A (f) – distinctiveness of appearance f  More can be learned from distinctive appearances (toxido on a campus) then from non-distinctive appearances (t-shirt & jeans on a campus)

8. Information-Theoretic appearance matching ► Distinctiveness weight for appearance f - AIC Weighting (Akaike ’ s Information Criterion) ► δ A – weighting coefficient ► Weighted match score

9. Algorithm Sketch ► Modelling Phase  Appearance density A(f) => D A (F)  inter-camera delay densities T i,j ► Estimation Phase  Computing evidence vectors  Inferring Transition Model from evidence vectors using Modified Multinomial Distribution

10. Modelling Phase ► Inter-Camera delay densities (function of time!) ► τ – the temporal window size ► Ψ = 1 if 0 < t2 – t1 <= τ, otherwise 0 ► K – is a smoothing or weighting kernel (such as a truncated Gaussian, a triangle filter, etc.) ► M(f 1,f 2 ) – match score defined above

11. Estimation Phase - 1 ► Observation o(t,f) weighting ► Normalized contribution vector w o, the j-th component expressing an estimate of the probability that the object observed in o appeared next in camera s.

12. Estimation Phase - 2 ► After m observations at camera c i, the node- specific evidence vector is given by: ► Higher weight is given to more distinctive appearances ► Last stage is to infer the transition model from the evidence vectors using Modified Multinomial Distribution..

13. Modified Multinomial Distribution ► An oral explanation

14. Stochastic Adaptive Tracking in a Camera Network Bi Song, Amit K.Roy-Chowdhury

15. Problem Formulation - 1 ► Network topology is assumed to be known (Can be the output of our previous algorithm..)  Connections between cameras  Entry/Exit points  Distribution of travel time between cameras. ► We want to be able to track multiple people routes between cameras

16. Problem Formulation - 2 ► n i c – a network node  i – node ’ s index  c – camera ► For any pair of nodes (n i c,n j d ), i != j: l i,j = 0 – if the two nodes are linked, 1 otherwise. ► Pτ(n i c,n j d ) – distribution of travel time between the two nodes.

17. Camera Network

18. Problem Formulation - 3 ► F n,t – observations at each node are represented as a feature vector ► F A – Normalized color (appearance feature) ► F I – Gait recognition (identity feature) ► Τ ni,nj t1,t2 – travel time between F ni,t1 and F nj,t2 ► F A, F I, T – independent random variables

19. Framework for Stochastic Adaptive Tracking

20. Feature Graph Construction - 1 ► The features are vertices on the graph (see next slide..) ► s(e ij ) is the similarity score between F i and F j Similarities computation takes into account known geometric and photometric transformations.

21. Feature Graph Construction - 2 ► The similarity score s ij is a realization of a random variable s ► The distribution of s on e ij is modeled as a normal distribution with s ij ’ as a mean: N(S ’ ij, σ ij 2 ) ► The variance is learned from the training data in an unsupervised manner.

22. Feature Graph Construction

23. Framework for Stochastic Adaptive Tracking

24. Optimal Path in Stochastic Feature Graphs - 1 ► Optimal path problem in graphs with weights as normal variables ► Von-Neumann and Morgenstern – define a utility function: it has to be monotonic, affine linear or exponential. ► We define a weighting function for similarity:

25. Optimal Path in Stochastic Feature Graphs - 2 ► We identify the most preferred set of paths by maximizing the utility function: ► This problem can be formulated as the maximum matching problem in a weighted bipartite graph: splitting each vertex into v in and v out and the weights are the utility function scores

26. Framework for Stochastic Adaptive Tracking

27. Path Smoothness Function ► PSF is defined on each edge e ij along its path λ q ► The feature vectors before e ij and after e ij are treated as two clusters: {X (0) } and {X (1) }

28. Closed-Loop Adaptation of Edge Similarities - 1 ► Whenever there is a peak in PSF function for some edge along a path, the validity of connections between features along that path is under doubt ► We adjust weight on the link where peak occurred by reducing mean, adjusting variance based on the learned values and recalculate the optimal paths using the new weights

29. Closed-Loop Adaptation of Edge Similarities - 2 ► Adaptation steps:

30. Closed-Loop Adaptation of Edge Similarities - 3 ► The process of weight adaptation and optimal path computation continues in a closed loop until a local minimum of is reached ► This process is repeated for each possible path λ q

31. Framework for Stochastic Adaptive Tracking

32. Stochastic Adaptive Tracking Algorithm - 1 ► 1. Construct a stochastic weighted graph where the vertices are the feature vectors and distribution of edge weights are set as described above. ► 2. Compute the optimal paths λ q ► 3. Compute PSF for each edge of λ q and adapt the distribution of edge weights.

33. ► 4. Repeat steps 2 and 3 until a local minimum of is reached. The final set of optimal paths is given as: Stochastic Adaptive Tracking Algorithm - 2

34. Questions? Thanks for your attention!