Distributed solutions for visual sensor networks to detect targets in crowds Cheng Qian

Outline Visual sensor networks for target detection Computing paradigms in sensor networks Local processing A centralized solution Distributed solution I Distributed solution II

Visual sensor networks The technology involves deploying (manually or from a plane) a large number of small, inexpensive motes over the area of interest. Each mote carries  Visual sensors ( CCD or thermal ) with limited range and field of view (FOV).  Limited computing capacities and storage resources.  Wireless channels to communicate with other sensors Why sensor networks for an application about target detection ?  Existence? A target may be occluded from the vision of “a” mote.  Localization? A visual mote is more like a orientation sensor.  3D Shape of the target? A mote only captures a 2D silhouette of the target. Collaboration among motes in the entire network.

Visual sensor networks Difference from a multi-perspective system where  each camera has no data processing capacity.  cameras are deployed with a planned strategy.  cameras are never enough to be called densely deployed  no energy and bandwidth concerns Key sentences in visual sensor networks for target detection  The local processing capacity should be fully exploited.  Sensors should be dynamically aware of the location of neighboring sensors  Decision should be made by integrating information captured by the entire network.  Redundant information should be discarded and data should be only directed to related sensors. A powerful central station Research Microsoft Trade-off between information integration and information transmission.

Information integration: entire network local motes Wireless transmission: long distance short distance Networking traffic jam: almost certainly almost impossible Visual sensor networks-Computing paradigms Illustrations from Xiaolin Wang’s thesis Centralized client/server modelDistributive peer to peer model Distributive cluster-based model Related motes are clustered, but how to define “motes being related”?? In the context of sensor networks, computing paradigm refers to the information processing model deployed in the application layer of the protocol stack.

Local processing in a Mote Each mote encloses useful information into limited bits. A target may be projected to a ridge in an image, and each ridge is represented by its central axis, height and colors. Raw image Silhouette Silhouette boundary Remove noises Remove spikes Ridges A mote

A centralized solution 1.Sweep the ridge through the common ground. and drop votes to spots the ridge passes by 2.Select the spot with highest number of compatible votes ( same color, same height…). This spot is declared to be occupied by a target. 3.Find all the ridges contributing to this declaration. Cancel all the other votes created by these ridges. Remove those spots between the declared spot and the ridges. 4.Go back to step 2. Integrating ridges from all the motes for detection and localization - A spot is seen having a ridge by multiple motes can be declared to be occupied by a target. The declaration gets more confirmed with the number of contributing motes increases. To save memory and increase running time, the ground plane is implemented by a quad-tree structure instead of a grid.

A centralized solution Shape reconstruction - A visual hull. Transmit all the critical points along the boundary of the ridge, and stack up the “slices” of visual hull.

Cluster related sensors A distributed solution I 1. Distance smaller than d d --- Ball Pivoting 2. FOV overlapped --- A convex hull. 3. Information is exchanged between neighboring clusters through motes on the cluster boundaries.

What about meshing the motes, and each mesh with three vertex motes forms a cluster? Each cluster is only responsible for detecting local occurrences inside that mesh. A distributed solution II Recall the centralized solution 1. Decomposed a central task requiring storage of a global map and computation about global optimization into local tasks belonging to each mesh. 2. Neighboring meshes can exchange information with each other to reduce redundancy.

Thanks