1. Assisted Target Detection for Search & Rescue
James Elder
Department of Computer Science and Engineering
Department of Psychology
&
Centre for Vision Research
York University

2. Project Overview
Improved sensing technologies have the potential to improve search and rescue performance.
The degree of improvement depends on how accurately the data generated by these technologies are interpreted, either by computer or by the human visual system.
The goal of the proposed project is to develop assisted target detection (ATD) algorithms that will improve human target detection rates by drawing visual attention to locations in the data stream where target probability is higher.
The SAR simulator at DRDC Toronto will serve as a test-bed for evaluating the efficacy of all image enhancements.
Project Start: Jan 1, 2006
Project Finish: OCE Funding expected to March 31, 2008

3. Project Team

4. York Search & Rescue ATD Team
Researcher
Position
% Research Time
Dr. James Elder PI
20%
Vladimir Magdin
M.Sc. Student
75%
Dr. Bob Hou
Senior Research Associate
Alumni
Researcher
Position
% Research Time
Dr. Francisco Estrada
Postdoctoral Fellow
10%
Kevin Cannons
PhD Student
20%

5. Industrial Partner Array Systems Computing
Specializes in computationally intensive, real-time digital signal processing systems using multiple parallel processors.
Provides high-performance software solutions in diverse application areas including sonar, radar and remote sensing, 3D modelling and simulation, and Software Defined Radio (SDR).
Located in Toronto, close to DRDC Downsview and York University.
Participants:
Rajesh Jha
Ray Obidowski
Steven Truelove

6. Funding Projection Contributors: OCE cash $8 0,000 $8 0,000 $8 0,0007 -
Dec -
05 to 1 -
Apr -
06 to 1 -
Apr -
07 to
Total
of all 31 -
Mar - 06 31 -
Mar - 07 31 -
Mar - 08
confirmed
(confirmed)
(propo
sed)
(proposed)
and proposed c
ontributions
by sponsor :
OCE
cash $8
0,000 $8
0,000 $8
0,000
$24
0,000
Array
cash $0 $0 $0 $ in -
kind $
68,400
$163,000
$163,000 $3
94,400
DRDC
cash $
50,000 $1 10
,000 $ 80
,000 $
240
,000 in -
kind
$20
9,000
$10,000
$10,000
$22
9,000
Total
contributions by $ 4 5
2,400 $
363 ,
000 $
333
,000
$1,
148
,400
project year:

7. Milestones Milestones Tasks Resources Start End Update Recruit
personnel
Recruit postdoctoral
fellow
Funding
Nov 05
Jan 06
Delayed due
to funding
delay
computer vision
graduate student Fu
nding
Feb 06 V.
Magdin
started Sept
Recruit psychophysics
Mar Y .
Morgenstern?
Complete
legal
agreements
Prepare and sign NDAs
and IP agreements
between York and
Array, allowing sharing
of source code.
Time with
institutional
lawyers
Dec
Completed

8. Milestones Milestones Tasks Resources Start End Update Establish
understan
ding
of problem
and
performance
benchmarks
Transfer C++ code for
indoor surveillance to
Array for integration
and testing on GSP
hardware.
Array
engineer
Dec 05
Mar 06
Completed
Review DND
documents on SAR and
AIMS, and review
general literature on
searc h -
rescue
Nov
Obtain existing search
rescue datasets with
ground truth for system
development and
evaluation
Existing DND
database 5

9. Milestones Milestones Tasks Resources Start End Update Single - frame
algorithm
development
and testing
Develop and evaluate
single
frame st
atistical
target models.
Dec 5
Jan 06
Completed
frame adaptive
background models.
Feb
Completed -
Discuss
Test frame by
ATD algorithm in
MATLAB on reserved
flight data using ROC
analysis
Mar
Evaluate statistical
against
AdaBoost methods
Jan 07
In Progress
Refine frame 07
Feb 07
Transfer MATLAB
code to Array for
integration and testing
on GSP hardware
Array engineer

10. Milestones Milestones Tasks Resources Start End Update Continuous
algorithm
development
and testing
Acquire new sear ch -
and
rescue datasets
with ground truth.
New DND
database
Apr 07
Mar
Develop or adapt
georeferencing
algorithms for search
rescue data
streams.
Georeferencing
tools from
AIMS project
Jun 07
Incorporate motion
cues and temporal
integratio
n into
foreground and
background models.
Jun
Jul 07
Test continuous ATD
algorithm in
MATLAB on reserved
flight data using ROC
analysis.
Aug
Refine continuous
Sept
Transfer MATLAB
code to Array for
integration and t
esting
on GSP hardware
Array engineer
Dec
Complete report on
ATD algorithms
Oct07
Complete publication
on ATD results

11. Milestones Milestones Tasks Resources Start End Update Develop and
evaluate
visualization
methods
Develop
candidate
for
overlaying
target
probability data onto
incoming visual data
stream
Dec 06
Mar 07
psychophysical
methodology for
evaluation using
DRDC SAR Simulator
Jun
Conduct human - in
the
loop SAR
simulator experiments
DRDC
simulator S
ept
Complete report on
methodologies and
results of SAR
simulation
experiments.
Sept
Complete publication
on SAR simulation
results 08

12. Milestones Milestones Tasks Resources Start End Update Final system
integration
and reporting
Refine complete
software system,
incl
uding ATD and
visualization
algorithms
Dec 07
Jan 08
Transfer final
MATLAB/C++ code
to Array for
integration and testing
on GSP hardware
Array engineer
Jan 08
Feb 08
Complete final report
on ATD system
Feb
Mar

13. Deliverables Assisted target detection algorithms
Assisted target detection software (MATLAB and/or C++)
A demonstration of ATD software integrated with Array Systems Scalable Generic Signal Processor technology for real-time operation.
Results of human-in-the-loop experiments using the ATD technology and the DRDC SAR simulation system.

14. York Progress Report

15. Ground Truth Labeling Labeled over 30,000 objects
Invested over 100 man-hours, split between 6 people
Labeled portions of all 7 video sequences that contain targets

16. Ground Truth – Example Frame
NIGHT EXERCISE_IR

17. Algorithm Evaluation
Selected subset of ground truth data from a single IR video (Night Exercise)
Not representative of search & rescue scenario
Training set
100 frames
5000 +ve examples
5000 -ve examples
Test set
185 frames
9000 targets

18. Algorithm Evaluation Hit Rate False Positives per Frame
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9 1 5 10 15 20
False Positives per Frame
Hit Rate
Pixel-wise bayes
Zero-mean Target Model
Normal Model
Static AdaBoost
Linear SVM

19. Cross-Validation
An effort is underway to evaluate the reliability of the ground truth data by cross-validating across human subjects.

23. Search and Rescue Simulator
York is running a copy of the DRDC Search & Rescue simulator
Do we need a Flybox?
May need to buy higher-end computer.