Paper Reading Dalong Du April.08, 2011

Paper Jilin Tu, Brandon Laften, Xiaoming Liu, Musodiq Bello, Jens Rittscher, Peter Tu. LPSM: Fitting Shape Model by Linear Programming

Outline Author Introduction. Problem Introduction. How to do? Discussion.

Outline Author Introduction. Abstract. Why to do? How to do? Discussion.

Author Introduction (1/2) Jilin Tu Xiaoming Liu

Author Introduction (2/2) Xiaoming Liu A Researcher in the Visualization and Computer Vision lab at GE Global Research.  His current research focuses on facial image processing in the context of surveillance videos. More specifically, He works on facial image alignment, face super-resolution from video sequences, face recognition, etc. Previously, He has also worked on multimedia information retrieval, video-based human animation, etc. In a broader sense, His interests include computer vision, pattern recognition, information retrieval, and machine learning. Education Background B.S. degree from Beijing Information Technology Institute, Beijing, China in 1997 M.S. degrees from Zhejiang University, Hangzhou, China in 2000 Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University, in 2004.

Outline Author Introduction. Abstract. Why to do? How to do? Discussion.

Abstract Introduction (1/2) Abstract—We propose a shape model fitting algorithm tha uses linear programming optimization. Most shape model fitting approaches (such as ASM, AAM) are based on gradient-descent-like local search optimization and usually suffer from local minima. In contrast, linear programming (LP) techniques achieve globally optimal solution for linear problems. In [1], a linear programming scheme based on successive convexification was proposed for matching static object shape in images among cluttered background and achieved very good performance. In this paper, we rigorously derive the linear formulation of the shape model fitting problem in the LP scheme and propose a LP shape model fitting algorithm (LPSM). In the experiments, we compared the performance of our LPSM with the LP graph matching algorithm(LPGM), ASM, and a CONDENSATION based ASM algorithm on a test set of PUT database. The experiments show that LPSM can achieve higher shape fitting accuracy. We also evaluated its performance on the fitting of some real world face images collected from internet . The results show that LPSM can handle various appearance outliers and can avoid local minima problem very well, as the fitting is carried out by LP optimization with l1 norm robust cost function.

Abstract Introduction (2/2) 摘要——我们提出了一种利用线性规划优化策略的形状模型匹配算法。大多数形状模型匹配算法（例如ASM，AAM）是基于类梯度下降局部搜索优化策略的，于是通常会陷入局部极小。与之相反的是，线性规划技术给出了针对线性问题的全局优化解决方案。[1]中提出了基于连续凸化的线性规划（LP）方案来匹配复杂背景下的静态对象形状，并取得了很好的效果。本文中，我们严格推导出了在LP方案下的形状模型匹配问题的线性形式，并提出了LP形状模型匹配算法（LPSM）。实验中，在PUT数据库的测试集上，我们对比了LPSM与LP图匹配算法（LPGM），ASM以及基于CONDENSATION的ASM算法的性能。实验表明，LPSM能够达到更高的形状匹配精度。我们也评价了算法在真实环境下人脸图像的性能，这些人脸图像从互联网上收集而来。结果显示LPSM能够处理各种各样的异常表观图像并且能非常好的避免局部极小问题，这是因为匹配是通过带有L1范式的鲁棒代价函数线性规划优化来完成的。

Outline Author Introduction. Abstract. Why to do? How to do? Discussion.

Why to do? (1/2) Brief summary Shape model Appearance model PCA->Kernel PCA, hierarchical Bayesian model Appearance model Gaussian distribution->mixture of Gaussians with discriminative features Fitting algorithm Coarse-to-fine iterative optimization process by PCA subspace->EM optimization, sampling based approach in a Bayesian framework Heavily depend on good initialization Gradient descent optimization in essence

Why to do? (2/2) Linear Programming Be able to achieve globally optimal solution to linear objective functions subject to linear constraint Solution Incorporate a shape subspace model into a LP framework

Outline Author Introduction. Abstract. Why to do? How to do? Discussion.

How to do?---pre-knowledge (1/4) Model Training A intrinsic shape S Shape alignment Procrustes Analysis PCA A extrinsic transformation θ similarity Transformation s, R, t Aligned Shapes c θ

How to do?---pre-knowledge (2/4) Intrinsic shape space to extrinsic image space Rearrange

How to do?---pre-knowledge (3/4) Intrinsic shape space to extrinsic image space, continue The training shape data is normalized by removing scale, rotation and translation Be Complementary Space for each other Rearrange We have

How to do?---pre-knowledge (4/4) Model fitting Fitting model to the object involves minimizing the following cost function with the constraint ,

How to do? (1/12) Graph matching by linear programming Given a point set , , pre-define a set of pair-wise neighboring model points is a target point in a query image matching to the point p The object function for minimizing the matching cost

How to do? (2/12) Graph matching by linear programming, continue

How to do? (3/12) Graph matching by linear programming, continue Remove L1 norm Use line segment to approximate unit circle 即 substitute with

How to do? (4/12) Shape model fitting by linear programming Formulate shape model in a LP framework

How to do? (5/12) Shape model fitting by linear programming, continue Linearization Scale factor object orientation Four line segment introduces inaccuracy Use polygon to approximate cycle

How to do? (6/12) Shape model fitting by linear programming, continue Object function algorithm

How to do? (7/12) Improvement Convex hull constraint in the intrinsic shape The PCA(hypercube) is not compact enough Add additional constraints by quantizing the shape space and modeling the allowable intrinsic shape parameter distributions by histograms in the top k most correlated dimensions in the parameter space Take k = 3

How to do? (8/12) Improvement Model appearance likelihood in maximal rejection classifier(MRC) subspace Carry out canny edge detection---blue point Evaluate the normalized correlation between the image patch on the landmark and the image patches centering on the canny edge points that are 4-10 pixels away from the landmark. Acquire eight image patches centering on the edges that have the maximal normalized correlation

How to do? (9/12) Implementation Obtain the Two method to compute the fitted landmarks

How to do? (10/12) Experiment PCA, FDA, MRC MRC more discriminative than PCA and more smoother than FDA

How to do? (11/12) Experiment

How to do? (12/12) Experiment

Outline Author Introduction. Abstract. Why to do? How to do? Discussion.

Discussion More powerful feature extraction method SIFT HOG Utilize more advantage learning techniques AdaBoost Make LPSM real-time Parallel computing Coarse-to-fine image pyramid analysis techniques

Thank you！