SVCL Automatic detection of object based Region-of-Interest for image compression Sunhyoung Han

SVCL Transmission in erroneous channel Basic Motivation Spatially different Super resolution Constraints Limited Resources & Channel Errors

SVCL Basic motivation By having information about importance of regions One can wisely use the limited resources

SVCL User-adaptive Coder v visual concepts of interest can be anything main idea: let users define a universe of objects of interest train saliency detector for each object e.g. regions of “people”, “the Capitol”, “trees”, etc.

SVCL User Adaptive Coder query provided by user train detector current training sets

SVCL User-adaptive coder user-adaptive coder: –detector should be generic enough to handle large numbers of object categories –training needs to be reasonably fast (including example preparation time) “face”“lamp”“car”

SVCL User-adaptive coder proposed detector –top-down object detector (object category specified by user) –focus on weak supervision instead of highly accurate localization –composed of saliency detection and saliency validation –discriminant saliency: saliency filters training FIND best features

SVCL Discriminant Saliency start from a universe of classes (e.g. “faces”, “trees”, “cars”, etc.) design a dictionary of features: e.g. linear combinations of DCT coefficients at multiple scales salient features: those that best distinguish the object class of interest from random background scenes. salient regions are the regions of the image where these detectors have strong response see [Gao & Vasconcelos, NIPS, 2004].

SVCL Top-down Discriminant Saliency Model Scale Selection W j WTA Faces Discriminant Feature Selection Salient Features Background Saliency Map Original Feature Set Malik-Perona pre-attentive perception model

SVCL saliency detector salient point sal i : –magnitude  i –location l i –scale s i saliency map approximated by a Gaussian mixture Saliency representation image saliency mapsalient points Probability map

SVCL Saliency validation saliency detection: –due to limited feature dictionary and/or limited training set –coarse detection of object class of interest need to eliminate false positives saliency validation: –geometric consistency –reject salient points whose spatial configuration is inconsistent with training examples original Image saliency map for ‘street sign’ example of saliency map

SVCL Saliency validation learning a geometric model of salient point con- figuration two components : - image alignment model: - classify points into true positives - configuration model false positives - model each as Gaussian

SVCL Saliency validation model: two classes of points Y={0,1} –Y=1 true positive –Y=0 false positive saliency map: mixture of true and false positive saliency distributions each distribution approximated by a Gaussian

SVCL this is a two class clustering problem –can be solved by expectation-maximization graphical model non-standard issues –we start from distributions, not points –alignment does not depend on false negatives Saliency validation E-stepM-step YX  L~uniform Y~Bernoulli (  1 ) C| Y=i ~multinomial (  i ) X| Y=i,L=l,S=s,  ~G(x, l- ,  ) L,S C

SVCL Saliency Validation For K training examples (# of saliency point is N k for kth example) Missing data  Y= j, j ∈ {1,0} Parameters   j (probability for class j) ∑ j (Covariance for class )  k (displacement for kth example) For robust update DERIVATION DETAILS

SVCL Saliency Validation visualization of EM algorithm Saliency detection result Init saliency points overlapped over 40 samples Visualized variance ∑ 1 Overlapped points classified as ‘’object’’ Overlapped points classified as ‘’noise’’ Visualized variance ∑ 0

SVCL Saliency Validation examples of classified Points in summary, during training we learn –discriminant features –The “right” configuration of salient points Examples of classified saliency points  White if h ij 1 >h ij 0 Black otherwise

SVCL Region of interest detection find image window that best matches the learned configuration mathematically: - find location p where the posterior probability of the object class is the largest

SVCL Region of interest detection by Bayes rule –Posterior  Likelihood x Prior –likelihood is given by matching saliencies within the window & the model - prior measures the saliency mass inside window ? ? likelihood Prior

SVCL Region of Interest Detection given the model –the likelihood, under it, of a set of points drawn from the observed saliency distribution is –and the optimal location is given by Prior for location P With saliency detector DERIVATION DETAILS Measure configuration matching

SVCL 2. Determine scale(shape) of ROI mask Observation(∑ * ) from data and prior(∑ 1 ) from training data are used 3. Thresholds P Y|X,P (1|x,p*) to get binary ROI mask Region of Interest Detection ** Once the center point is known the assignment of each point is given by  The observed configuration for Y=1 is x ∑1∑1 ∑*∑*

SVCL Region of Interest Detection Saliency detection (for statue of liberty) Probability map (saliency only) Probability map (with configuration info.) ROI mask Example of ROI Detection

SVCL Evaluation Using CalTech “Face” database & UIUC “Car side” database Evaluate robustness of learning –Dedicated Training set vs. Web Training set Evaluation Metric –ROC area curve –PSNR gain for ROI coding vs. normal coding Number of positive example: 550 Number of positive example: 100

SVCL Evaluation ROC area curve False Positive True Positive False Positive True Positive “Car” “Face”

SVCL Evaluation PSNR performance comparison “Car”“Face” Bit Per Pixel PSNR Bit Per Pixel PSNR 14.3% bits can be saved even with web train uniform case for the same image quality

SVCL Result Examples

SVCL Result Comparison of needed bits to get the same PSNR (30 dB) for ROI  Maximally, ¼ bits are enough to get the same quality for ROI area

SVCL Result Examples Normal coding ROI coding

SVCL

EM derivation Want to fit lower level observation For a virtual sample X = {X ik |i=1, …, N k and k=1, …, K} with the size of M ik = ik *N, likelihood becomes For complete set the log likelihood becomes 

SVCL EM derivation  Maximization in the m-step is carried out by maximizing the Lagrangian

SVCL ROI Detection For one sample point x 1  For samples having distribution of

SVCL ROI Detection Therefore,