1. Applications of
Shape Similarity

2. Robotics: Shape Screening
ASR: Applications in Computer Vision
Robotics: Shape Screening
(Movie: Robot2.avi)
Straightforward Training Phase
Recognition of Rough Differences
Recognition of Differences in Detail
Recognition of Parts

3. View Invariant Human Activity Recognition
ASR: Applications in Computer Vision
Application 2:
View Invariant Human Activity Recognition
(Dr. Cen Rao and Mubarak Shah, School of Electrical Engineering and Computer Science, University of Central Florida)

4. Human Action Defined by Trajectory
Application: Human Activity Recognition
Human Action Defined by Trajectory
Action Recognition by Comparison of Trajectories
(Movie: Trajectories)
Rao / Shah:
Extraction of ‘Dynamic Instants’ by Analysis of Spatiotemporal Curvature
Comparison of ‘Dynamic Instants’ (Sets of unconnected points !)
ASR:
Simplification of Trajectories by Curve Evolution
Comparison of Trajectories

5. Application: Human Activity Recognition
Simplification
Trajectory

6. Activity Recognition: Typical Set of Trajectories

7. Trajectories in Tangent Space

8. Trajectory Comparison by ASR: Results

9. Recognition of 3D Objects by Projection
Background: MPEG 7 uses fixed view angles
Improvement: Automatic Detection of Key Views

10. Automatic Detection of Key Views
(Pairwise) Comparison of Adjacent Views
Detects Appearance of Hidden Parts

11. Automatic Detection of Key Views
Result (work in progress):

12. The Database Implementation
Application: ASR
The Database Implementation

13. The Main Application: Back to ISS
Task:
Create Image Database
Problem:
Response Time
Comparison of 2 Shapes: 23ms on Pentium1Ghz
ISS contains 15,000 images:
Response Time about 6 min.
Clustering not possible:
ASR failed on measuring dissimilarities !

14. Vantage Objects Solution:
Full search on entire database using a simpler
comparison
Vantage Objects (Vleugels / Veltkamp, 2000) provide a simple comparison of
n- dimensional vectors (n typically < 100)

15. Vantage Objects The Idea:
Compare the query-shape q to a predefined subset S of the shapes in the database D
The result is an n-dimensional Vantage Vector V,
n = |S| s1 v1 s2 v2 q s3 v3 … sn vn

16. Vantage Objects
- Each shape can be represented by a single Vantage Vector
- The computation of the Vantage Vector calls the ASR – comparison only n times
- ISS uses 54 Vantage Objects, reducing the comparison time (needed to create the Vantage Vector) to < 1.5s
- How to compare the query object to the database ?

17. Vantage Objects
- Create the Vantage Vector vi for every shape di in the database D
- Create the Vantage Vector vq for the query-shape q
- compute the (euclidean) distance between vq and vi
- best response is minimum distance
Note: computing the Vantage Vectors for the database objects is an offline process !

18. How to define the set S of Vantage Objects ?

19. k=1..i-1 e(di , sk) maximal. (e = eucl. dist.)
Vantage Objects
Algorithm 1 (Vleugels / Veltkamp 2000):
Predefine the number n of Vantage Objects
S0 = { }
Iteratively add shapes di  D\Si-1 to Si-1 such that
Si = Si-1  di
and
k=1..i-1 e(di , sk) maximal. (e = eucl. dist.)
Stop if i = n.

20. Vantage Objects
Result:
Did not work for ISS.

21. Algorithm 2 (Latecki / Henning / Lakaemper):
Vantage Objects
Algorithm 2 (Latecki / Henning / Lakaemper):
Def.:
A(s1,s2): ASR distance of shapes s1,s2
q: query shape
‘Vantage Query’ : determining the result r by minimizing e(vq , vi ) vi = Vantage Vector to si
‘ASR Query’: determining the result r by minimizing A(q,di )
Vantage Query has certain loss of retrieval quality compared to ASR query.
Define a loss function l to model the extent of retrieval performance

22. Vantage Objects
Given a Database D and a set V of Vantage Vectors, the loss of retrieval performance for a single query by shape q is given by:
lV,D (q) = A(q,r),
Where r denotes the resulting shape of the vantage query to D using q.
Property:
lV,D (q) is minimal if r is the result of the ASR-Query.

23. L(S) = 1/n  lS,D\{si} (si)
Vantage Objects
Now define retrieval error function L(S) of set
S={s1 ,…, sn }  D of Vantage Vectors of Database D:
L(S) = 1/n  lS,D\{si} (si)
Task:
Find subset S  D such that L(S) is minimal.

24. Vantage Objects Algorithm: V0={ }
iteratively determine sj in D\Sj-1 such that
Sj =Sj-1  sj and L(Vj) minimal.
Stop if improvement is low

25. Number of Vantage Objects
Result:
Worked fine for ISS, though handpicked objects still performed better.
Handpicked
Algorithm 2
L(S)
Number of Vantage Objects

26. Vantage Objects
…some of the Vantage Objects used in ISS:

27. Vantage Objects and ISS
The Vantage Objects are used in the ASR in the first (handdrawn) query.
The query is compared to 54 Objects, then a vector comparison is computed with the whole database.
The first result, also called ‘first guess’, is the result of the vantage vector search.
Searching for a ‘grabbed’ a shape on the user interface leads to direct comparison with the ASR, these results are precomputed, since the query is a known shape !

28. Vantage Objects and ISS
A: the handdrawn sketch
B: the result of the Vantage search
C: the result of the exact match