Distributed solutions for visual sensor networks to detect targets in crowds

Cheng Qian

Outline

• Visual sensor networks for target detection • Computing paradigms in sensor networks • Local processing • A centralized solution • Distributed solution I • Distributed solution II

Visual sensor networks

The technology

involves deploying (manually or from a plane) a large number of small, inexpensive motes over the area of interest.

Each mote carries

 Visual sensors ( CCD or thermal ) with limited range and field of view (FOV).  Limited computing capacities and storage resources.  Wireless channels to communicate with other sensors www.ece.ncsu.edu

Why sensor networks for an application about target detection ?

 Existence? A target may be occluded from the vision of “a” mote.  Localization? A visual mote is more like a orientation sensor.  3D Shape of the target? A mote only captures a 2D silhouette of the target. Collaboration among motes in the entire network .

Research Microsoft

Visual sensor networks

Difference from a multi-perspective system where

 each camera has no data processing capacity.  cameras are deployed with a planned strategy.  cameras are never enough to be called densely deployed  no energy and bandwidth concerns A powerful central station

Key sentences in visual sensor networks for target detection

 The local processing capacity should be fully exploited.  Sensors should be dynamically aware of the location of neighboring sensors  Decision should be made by integrating information captured by the entire network.  Redundant information should be discarded and data should be only directed to related sensors. Trade-off between information integration and information transmission.

Visual sensor networks-Computing paradigms

In the context of sensor networks, computing paradigm refers to the information processing model deployed in the application layer of the protocol stack.

Centralized client/server model Distributive peer to peer model

Information integration

: entire network local motes

Wireless transmission

: long distance short distance

Networking traffic jam

: almost certainly almost impossible

Distributive cluster-based model

Related motes are clustered, but how to define “motes being related”??

Illustrations from Xiaolin Wang’s thesis

Local processing in a Mote

Each mote encloses useful information into limited bits.

Raw image Silhouette A mote Silhouette boundary Remove noises Remove spikes Ridges A target may be projected to a ridge in an image, and each ridge is represented by its central axis, height and colors.

A centralized solution

Integrating ridges from all the motes for detection and localization -

A spot is seen having a ridge by multiple motes can be declared to be occupied by a target. The declaration gets more confirmed with the number of contributing motes increases. 1.

2.

3.

4.

Sweep the ridge through the common ground. and drop votes to spots the ridge passes by Select the spot with highest number of compatible votes ( same color, same height…). This spot is declared to be occupied by a target.

Find all the ridges contributing to this declaration. Cancel all the other votes created by these ridges. Remove those spots between the declared spot and the ridges.

Go back to step 2. To save memory and increase running time, the ground plane is implemented by a quad-tree structure instead of a grid.

A centralized solution

Shape reconstruction - A visual hull.

Transmit all the critical points along the boundary of the ridge, and stack up the “slices” of visual hull.

A distributed solution I

Cluster related sensors

1. Distance smaller than

d

--- Ball Pivoting

d

2. FOV overlapped --- A convex hull. 3. Information is exchanged between neighboring clusters through motes on the cluster boundaries.

A distributed solution II

Recall the centralized solution What about meshing the motes, and each mesh with three vertex motes forms a cluster? Each cluster is only responsible for detecting local occurrences inside that mesh. 1. Decomposed a central task requiring storage of a global map and computation about global optimization into local tasks belonging to each mesh. 2. Neighboring meshes can exchange information with each other to reduce redundancy.

Thanks