1. Paper Reading
Dalong Du
April.08, 2011

2. Paper
Jilin Tu, Brandon Laften, Xiaoming Liu, Musodiq Bello, Jens Rittscher, Peter Tu. LPSM: Fitting Shape Model by Linear Programming

3. Outline Author Introduction. Problem Introduction. How to do?
Discussion.

4. Outline Author Introduction. Abstract. Why to do? How to do?
Discussion.

5. Author Introduction (1/2)
Jilin Tu
Xiaoming Liu

6. Author Introduction (2/2)
Xiaoming Liu
A Researcher in the Visualization and Computer Vision lab at GE Global Research. 
His current research focuses on facial image processing in the context of surveillance videos. More specifically, He works on facial image alignment, face super-resolution from video sequences, face recognition, etc. Previously, He has also worked on multimedia information retrieval, video-based human animation, etc. In a broader sense, His interests include computer vision, pattern recognition, information retrieval, and machine learning.
Education Background
B.S. degree from Beijing Information Technology Institute, Beijing, China in 1997
M.S. degrees from Zhejiang University, Hangzhou, China in 2000
Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University, in 2004.

7. Outline Author Introduction. Abstract. Why to do? How to do?
Discussion.

8. Abstract Introduction (1/2)
Abstract—We propose a shape model fitting algorithm tha uses linear programming optimization. Most shape model fitting approaches (such as ASM, AAM) are based on gradient-descent-like local search optimization and usually suffer from local minima. In contrast, linear programming (LP) techniques achieve globally optimal solution for linear problems. In [1], a linear programming scheme based on successive convexification was proposed for matching static object shape in images among cluttered background and achieved very good performance. In this paper, we rigorously derive the linear formulation of the shape model fitting problem in the LP scheme and propose a LP shape model fitting algorithm (LPSM). In the experiments, we compared the performance of our LPSM with the LP graph matching algorithm(LPGM), ASM, and a CONDENSATION based ASM algorithm on a test set of PUT database. The experiments show that LPSM can achieve higher shape fitting accuracy. We also evaluated its performance on the fitting of some real world face images collected from internet . The results show that LPSM can handle various appearance outliers and can avoid local minima problem very well, as the fitting is carried out by LP optimization with l1 norm robust cost function.

9. Abstract Introduction (2/2)
摘要——我们提出了一种利用线性规划优化策略的形状模型匹配算法。大多数形状模型匹配算法（例如ASM，AAM）是基于类梯度下降局部搜索优化策略的，于是通常会陷入局部极小。与之相反的是，线性规划技术给出了针对线性问题的全局优化解决方案。[1]中提出了基于连续凸化的线性规划（LP）方案来匹配复杂背景下的静态对象形状，并取得了很好的效果。本文中，我们严格推导出了在LP方案下的形状模型匹配问题的线性形式，并提出了LP形状模型匹配算法（LPSM）。实验中，在PUT数据库的测试集上，我们对比了LPSM与LP图匹配算法（LPGM），ASM以及基于CONDENSATION的ASM算法的性能。实验表明，LPSM能够达到更高的形状匹配精度。我们也评价了算法在真实环境下人脸图像的性能，这些人脸图像从互联网上收集而来。结果显示LPSM能够处理各种各样的异常表观图像并且能非常好的避免局部极小问题，这是因为匹配是通过带有L1范式的鲁棒代价函数线性规划优化来完成的。

10. Outline Author Introduction. Abstract. Why to do? How to do?
Discussion.

11. Why to do? (1/2) Brief summary Shape model Appearance model
PCA->Kernel PCA, hierarchical Bayesian model
Appearance model
Gaussian distribution->mixture of Gaussians with discriminative features
Fitting algorithm
Coarse-to-fine iterative optimization process by PCA subspace->EM optimization, sampling based approach in a Bayesian framework
Heavily depend on good initialization
Gradient descent optimization in essence

12. Why to do? (2/2) Linear Programming
Be able to achieve globally optimal solution to linear objective functions subject to linear constraint
Solution
Incorporate a shape subspace model into a LP framework

13. Outline Author Introduction. Abstract. Why to do? How to do?
Discussion.

14. How to do?---pre-knowledge (1/4)
Model Training
A intrinsic shape S
Shape alignment
Procrustes Analysis
PCA
A extrinsic transformation θ
similarity Transformation
s, R, t
Aligned Shapes c θ

15. How to do?---pre-knowledge (2/4)
Intrinsic shape space to extrinsic image space
Rearrange

16. How to do?---pre-knowledge (3/4)
Intrinsic shape space to extrinsic image space, continue
The training shape data is normalized by removing scale, rotation and translation
Be Complementary Space for each other
Rearrange
We have

17. How to do?---pre-knowledge (4/4)
Model fitting
Fitting model to the object involves minimizing the following cost function
with the constraint ,

18. How to do? (1/12) Graph matching by linear programming
Given a point set , , pre-define a set of pair-wise neighboring model points
is a target point in a query image matching to the point p
The object function for minimizing the matching cost

19. How to do? (2/12)
Graph matching by linear programming, continue

20. How to do? (3/12) Graph matching by linear programming, continue
Remove L1 norm
Use line segment to approximate unit circle
即 substitute with

21. How to do? (4/12) Shape model fitting by linear programming
Formulate shape model in a LP framework

22. How to do? (5/12) Shape model fitting by linear programming, continue
Linearization
Scale factor
object orientation
Four line segment introduces inaccuracy
Use polygon to approximate cycle

23. How to do? (6/12) Shape model fitting by linear programming, continue
Object function
algorithm

24. How to do? (7/12) Improvement
Convex hull constraint in the intrinsic shape
The PCA(hypercube) is not compact enough
Add additional constraints by quantizing the shape space and modeling the allowable intrinsic shape parameter distributions by histograms in the top k most correlated dimensions in the parameter space
Take k = 3

25. How to do? (8/12) Improvement
Model appearance likelihood in maximal rejection classifier(MRC) subspace
Carry out canny edge detection---blue point
Evaluate the normalized correlation between the image patch on the landmark and the image patches centering on the canny edge points that are 4-10 pixels away from the landmark.
Acquire eight image patches centering on the edges that have the maximal normalized correlation

26. How to do? (9/12) Implementation Obtain the
Two method to compute the fitted landmarks

27. How to do? (10/12) Experiment PCA, FDA, MRC
MRC more discriminative than PCA and more smoother than FDA

28. How to do? (11/12)
Experiment

29. How to do? (12/12)
Experiment

30. Outline Author Introduction. Abstract. Why to do? How to do?
Discussion.

31. Discussion More powerful feature extraction method
SIFT
HOG
Utilize more advantage learning techniques
AdaBoost
Make LPSM real-time
Parallel computing
Coarse-to-fine image pyramid analysis techniques

32. Thank you！