1. Slow moving point target Tracking & Detecting in real and compressed infrared imagery sequences
Students
Liav Viner
Omri Ravid
Supervisors
Dr. Ofer Hadar
Mrs. Revital Huber-Shalem

2. Motivation Where is the target..?
Infrared imagery sequences are used for detection of moving targets in the presence of evolving cloud clutter or background noise
This research concentrates on slow moving point targets, which are one pixel in size, such as long-range aircraft

3. Point target IR movies compression and detection system
Motivation
Infrared sequences:
- Captured by ground sensors
- Contain enormous amount of data
Transmission and storage are very time and resource consuming
=>Compression which retains the point target detection capabilities
Point target IR movies compression and detection system

4. Compression
IR compression method containing point targets cannot use a video compression standard
MPEG
used for video streams watched by human viewers
Exploit the human visual system characteristics
Different measures of quality than IR movies containing point targets
A point target cannot be detected in IR movies by a human viewer,
Target size is only one pixel
Target intensity is not always noticeable within the background

5. Compression methods Spatial 2-D DCT
Can we use a lossy spatial 2-D DCT compression?
DCT Transform
ZigZag
Vector
Compression
IDCT Transform
Most of the data is in the low-frequency coefficients and the DC itself, and most of the AC coefficients are negligible
Low-frequency coefficients represent slow spatial changes, while high-frequency coefficients represent fast spatial changes

6. Background Temporal profile
Temporal profile is the temporal behavior of a pixel along the movie frames
Target pixel
Clear sky pixel
The intensity of the point target is proportional to the height of the peak
The velocity of the point target is inversely proportional to the width of the peak
Cloud edge pixel

7. Compression methods Spatial 2-D DCT
Main drawback of a standard compression for IR imagery containing point targets:
lossy spatial compression (as spatial DCT),
reduces high spatial frequencies
might diminish the point target
Each frame has DCT coefficients
64,000
protected coefficients
16,000
protected coefficients
1,000
protected coefficients
100
protected coefficients
Compression rate: 1.22
Compression rate: 4.88
Compression rate: 78.08
Compression rate: 780.8
Temporal profile of
a Target’s pixel

8. Compression methods Temporal 1-D DCT
The 1-D DCT-based lossy temporal compression method is more suitable for IR movies containing a slow moving point target
4th Coefficient
1st frame
2nd frame
3rd frame
4th frame
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
3rd Coefficient
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
2nd Coefficient
1st Coefficient (DC)
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
DCT coefficients
of pixel (x,y)
Target
Noise
Temporal profile
This compression is supposed
to reduce the temporal noise and
compress the temporal profile
Temporal profile after
Temporal DCT compression

9. Compression methods Temporal 1-D DCT
How many coefficients should we protect?
Movie score
Compression Rate
Number of Protected Coefficients
Number of Protected Coefficients

10. Compression methods Temporal 1-D DCT
Number of Protected Coefficients
How does the elimination of coefficients affects the temporal profile of a target pixel?
Compression rate 1 2 5 9 13 95
47.5 19
10.55
7.3

11. Compression methods Temporal 1-D DCT
First two coefficients (including DC) do not contain target information but more background value
Target appears in coefficients range [3,13]
Coefficients larger than 13 contain mainly fast changes and noise information
1/95 coefficient
2/95 coefficient
5/95 coefficient
9/95 coefficient
13/95 coefficient

12. Compression methods Quantization
After choosing the number of protected coefficients, we can now use Lloyd-Max quantization for further compression
Lloyd Max
Quantization
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
Code book
3700
11 = 3350
10 = 2000
01 = 750
00 = 250
3000
As the number of levels decrease, the target becomes less noticeable
1000
500
40 Levels
10 Levels
4 Levels

13. Spatial 2-D DCT Compression summery Temporal 1-D DCT
Spatial 2-D DCT compression performance becomes poor above compression rate of 3
Temporal 1-D DCT compression performance becomes poor above compression rate of 12
Further compression on the Temporal 1-D DCT’s output with Max-Lloyd Quantization improves the performance even better
Spatial
2-D DCT
Temporal
1-D DCT

14. Point Target Tracking & Detecting
VERS - Variance Estimation Ratio Score
SNR based
Evaluates the final score of
each sub-temporal profile
Temporal process compares the overall estimated variance of a temporal profile and its highest fluctuation
If this fluctuation is high enough,
it may be considered as a target

15. VERS background Scores
The ratio between the maximum variance and the average of the K lowest variances is the score of the pixel
Short-Term
Variance Window
The variance is calculated over shorter time sections after subtracting the DC from the partial temporal profile
Pixel Score(i)=
Block Score(i)=
Pixel profile
Movie Score(i)=
Variance profile

16. VERS background Parameters
The VERS algorithm has another important parameter:
Long-term DC window
The long-term window for DC estimation should be short enough in order to track DC changes caused by a clutter entrance or departure, but long enough to perform an accurate estimation to suppress noise and not to suppress the target
Long-Term DC Window
Edge cloud
Point target
Pixel Profile
DC Profile
Result

17. VERS background Parameters
The long-term DC window’s size has a great effect on the movie’s score
Too small – Might erase target
Too big - Might suspect cloud as a target
Temporal profiles of a cloud
Window Size - 20
Window Size - 30
Pixel Profile
DC Profile
Result
Mistakenly suspect as target

18. VERS background Movies scores
VERS shortcomings:
Dependency of the performance and the chosen parameters on the unknown target velocity and scene's type
Movie’s score VS Set of Parameters
5/15
5/20
5/25
5/30
7/30
7/35
7/37
7/50
10/30
10/40
10/50
30/35
35/50
j2a
53.189
78.538
93.203
98.532
96.868
74.086
84.321
78.420
59.288
37.326
13.755
j13c
3.215
4.655
5.481
5.959
7.298
10.519
11.694
8.991
8.450
11.656
8.252
2.796
1.017
m21f
6.357
7.337
7.553
6.173
5.934
5.307
6.193
6.530
5.945
6.743
6.922
9.707
11.169
na23a
19.738
24.580
23.095
17.658
18.092
13.810
11.421
5.718
16.289
7.940
4.504
1.866
0.289
npa
30.183
36.422
40.568
36.051
38.674
38.848
46.336
59.552
34.442
45.023
58.046
21.304
15.881
But what distinguishes between each movie?

19. Scene’s Type
Determining the scene's type automatically will allow us to select the optimal long-term DC window parameter for the given IR sequence
- Rapidly evolving clutter Shorter DC window
- Slow evolving clutter, or Clear Sky - Longer DC window
Two different methods for determining the scene's type:
Spatial 2-D DCT
Tail Ratio
Hot hazy night
Wispy clouds
Bright clouds
Fluffy clouds
Bright clouds
Cloudy

20. Determining Scene’s Type Spatial 2-D DCT
We are interested in knowing the scene's type, particularly knowing if the scene has many spatial changes, implicating it is composed from a large amount of small clouds
First approach for determining the scene's type: Spatial 2-D DCT on the first frame of the IR sequence

21. Determining Scene’s Type Spatial 2-D DCT
1’st Frame
DCT Vector
Applying spatial 2-D DCT on a frame of cloudy sky should result in many large coefficients for all frequencies
Applying spatial 2-D DCT on a frame which is mainly composed of homogenous scene, i.e. clear sky or very big clouds, should result in more negligible coefficients

22. Determining Scene’s Type Spatial 2-D DCT
Summing up all DCT coefficients yields a value which implies on the scene’s type
633.7
816.1
1564.7
1845.5
1894.2
2054.7
Homogenous Scene
clutter Scene

23. Determining Scene’s Type Tail Ratio
Second approach for determining the scene's type: The Tail Ratio method
1st Frame
Histogram
Assumption of work: many small clouds result in a large variety of histogram values, each having a small amount of occurrences, and therefore visualized as a low bin in the histogram graph
This phenomenon results in a descending tale of higher intensities than the sky's

24. Determining Scene’s Type Tail Ratio
We have generated a MATLAB function that counts all the histogram bins within a certain range 31 42
159
225
237
415
Homogenous Scene
clutter Scene
The scene might change for longer movies
Thus, both of the techniques should be applied periodically in case of longer movies

25. Determining Scene’s Type Automatic Method
Spatial DCT
Tail Ratio
Clutter Level 1 2 3 4 5
Automatic Method categorizes the scene into one of five different levels of scene’s type
Optimal Long-Term DC Window for the selected scene’s type
Higher movie’s score

26. Path Tracking Algorithm PTA
The resulting output of the VERS algorithm on an infrared sequence is an image representing each pixel’s score
It is not necessarily true that the pixel
with the highest value is the target
So, we need to track the pixels
through which the target traversed
Cloud
VERS
129
172
121
108
170
162
100
163
124
130
524
382
101
148
201
408
123
667
623
429
362
167
182
140
132
149
141
152
120
138
154
136
Target
PTA

27. PTA The Algorithm
Potential_Targets= all pixels that received high score (according to PS threshold)
Declare each pixel from Potential_Targets as a Path Group (PG)
For each PG
Search for neighboring PGs (according to DBP threshold)
If found a neighboring PG
Combine both PGs to one PG
Else go to 9
Go to 3
If PG has less pixels than PGS threshold
Erase PG
If there is no PGs left
Declare the pixel with the highest score as target
If there are neighboring pixels within PG target
Combine pixels to the PG target
Mark each PG as a different target's track
Verify that the resulting path represents a real and logical movement of an aircraft

28. PTA Algorithm’s Steps
Original IR sequence

29. PTA Algorithm’s Steps
All scored pixels (result of VERS) are colored in yellow

30. PTA Algorithm’s Steps
Pixels Score (PS) threshold filtering low-scored pixels

31. PTA Algorithm’s Steps
Initializing each pixel as a group

32. PTA Algorithm’s Steps
Join neighboring groups into same group using the Distance Between Pixels (DBP) threshold

33. PTA Algorithm’s Steps
Delete all groups that contain less pixels than the Paths Group's Size (PGS) threshold

34. PTA Algorithm’s Steps
Verify that the resulting path represents a real and logical movement of an aircraft

35. PTA Algorithm’s Steps
Verify that the resulting path represents a real and logical movement of an aircraft

36. Conclusions Compression of IR imagery containing point target
- Lossy temporal compression yields better point-target detection
results than lossy spatial compression
- Further compression using Max-Lloyd Quantization improves the
performance even better
VERS improvement
- Finding an optimal set of VERS parameters
- Determining the scene’s type
 Matching optimal DC window
 Yields better results
Path Tracking Algorithm (PTA)
- The target’s track can be detected
even if not all target pixels achieved high pixel scores

37. Any Questions..?