1. Video and Sensor Network Architecture and Displays
Douglas Fidaleo
Mohan Trivedi (PI)
Center for Wireless
COMMUNICATIONS

2. Overview
Sensor data can be overwhelming especially with visual (video) sensors
Transmission of higher level semantic content rather than low level detail
Network architecture in support of this goal

3. Network Architecture: Goals
Data warehouse
Flexibility
Reconfigurability
Easily Scalable
Bi-directional communication
Simulated sensor-to-sensor communication
Sensors
2-3 min
Scalability is especially important for sensor networks as there are so many types of sensors: GPS, Text message from PDA, seismic, temperature, fluid flow, video. And they come in totally different packages: Cameras come on cell phones, PDAs, fixed video feeds, controllable PTZ cameras.
Bidirectionality: Most sensor networks are designed to collect information from point sources and transmit to a single data warehouse. These architectures don’t allow for adaptaptive sensing of the environment. For example, each local node likely has little analytical power, so most of the analysis is occuring off site…but what if that off-site processor determines that some change in sensing will make the information more useful…we need to be able to communicate that back to the sensor. A simple example of this is the PTZ unit of a video sensor. If the analysis module determines that there is something of interest in the scene, it should be able to send a control signal to the PTZ unit to allow it to focus in on the event. This is also essential for selective capture of events…for example, for high power or bandwidth sensors, we may not want to or be able to consistently capture information…we can hoewever, use a control signal to trigger the capture at times that are most relevant.

4. Observation Application
Network Architecture
Clients
Database
Application
Observation Application
GPS
Cell phone
Tablet PC
Seismic sensor
General framework
Thin clients
XML data packets
Bi-directional communication
Independent video server
Video Analysis
Application
Server
(2-3 min)
Describe general architecture: thin clients, XML packets, bidirectional,
Explain how different modules can be instantiated as examples of clients
Explain how video must be handled differently as it is streaming data
Show direct video server connections. Clients can feed extracted events onto the network.

5. Mobile Visual Communicator
Clients
Clients
Database
Application
Observation Application
GPS
Cell phone
Tablet PC
Seismic sensor
Tablet PC
Wireless transmission and visualization of user position
Remote mobile video observations
snapshots
GPS
Video Analysis
Application
Server
Server
(2-3 min)
Describe general architecture: thin clients, XML packets, bidirectional,
Explain how different modules can be instantiated as examples of clients
Explain how video must be handled differently as it is streaming data
Show direct video server connections. Clients can feed extracted events onto the network.

6. Demo application:

7. Current Implementation
Server built on top of Fortress online game server
Secure client authentication
Secure communication channels
Added sensor network Java API
Thin client
Data descriptors
Message routing and filtering
Current implementation uses the Fortress online game server as the glue to provide basic secure client authentication and packet handling. We have developed a layer on top of this to handle client message filters and routing of packets. We have also developed a set of thin clients in Java for interfacing to the server.

8. Visualization and Interfaces
Direct visualization of data streams
Temperature gauge
Embedded visualization
GPS location on a spatial map
Video augmented 2D maps
Data abstraction and interpretation
The NeST is intended to be a flexible backbone for event capture, but there are other challenges such visualization of the data streams.
Again there are several different levels at which we can approach this: T
A very important area of exploration is how to abstract and make sense of the abundance of sensor data. If we have a mish-mash of thousands of sensors, it is clearly impossible to directly visualize each sensor, whether they are embedded or not. For certain domains it may be possible to simply pick out certain sensors that are problematic with a beacon, but we may require more in-depth understanding.

9. Data Abstraction: The SimCity Analogy
City is populated with sensors
Advisors make sense of data
Provide feedback to operator (Mayor)
An interesting analogy here are the “advisors” in the game SimCity. The game is effectively a set of virtual sensors, and the advisor’s role is to make sense of the information and point out to the Mayor the areas that must be addressed. The advisor’s each have specific roles and goals, but their primary function is to interface between the abundant sensor data and the “operator”.

10. Future Plans: Short Term
Security and Privacy
Client-server architecture enhancement
Sensor control API
Application APIs
Interface existing capture and analysis modules
Face recognition, Body posture analysis, Body tracking
Integrate a wider variety of sensors and processing clients
RFIDs, PDAs, Cell phones
Security: Can easily secure the connection channels between the clients and the server, but also need a way to restrict access to data
Privacy: User selectable tracking is a first step. But the issue is more complicated. Object tracking generally occurs before identification, so there may be an implicit record of where you were for the last 5 minutes, before the system actually knows that this record belongs to you. So you can say “only track those objects for whom you have the identity”, but this excludes a huge portion of potentially useful information.

11. Future Plans: Longer Term
Large scale data abstraction and human interfacing (aka the SimCity model)
Sensor data archiving (DB)