PLS Regression Hervé Abdi The university of Texas at Dallas

An Example: What is Mouthfeel? From Folkenberg D.M., Bredie W.L.P., Martend M., (1999). What is mouthfeel: Sensory-rheological relationship in instant hot cocoa drinks. Journal of Sensory Studies, 14, (Data set courtoisie of Marten, H., Marten M. (2001) Multivariate Analysis of Quality: An introduction. London: Wiley. Downloaded from: Data set: Cocoa-ii.mat Goal. Predict sensory attributes (mouthfell): Dependent variables (Y set) from physical/chemical/rheological properties: Predictors / independent variables (X set)

An Example: What is Mouthfeel? 6 Predictors / independent variables (X set) physical/chemical/rheological properties %COCOA %SUGAR %MILK SEDIMENT COLOUR VISCOSITY 10 Dependent variables (Y set) colour cocoa-odour milk-odour thick-txtr mouthfeel smooth-txtr creamy-txtr cocoa-taste milk-taste sweet 14 Samples (n-: without stabilizer, n+: are with stabilizer)

X

Y

Why using PLS and PCA and MLR A short tour

I by J data sets: PCA, CA, Biplots, etc. I J The beauty of Euclide …

I by J  I by 1 (with J << I) data sets: Multiple Regression IJ1 The beauty of Euclide

I by J  I by K data sets: PLS, CANDIS, etc. IJK The beauty of Euclide

Why using PLS ? 1.To explain the similarity between the observations (here cocoa samples). 2.To detect the structure in the relationships between dependent and independent variables. 3.To get a graphical representation of the data 4.To predict the value of new observations

PLS combines features of Principal Component Analysis (PCA) and Multiple Linear Regression (MLR). Like PCA: PLS extracts factors from X. Like MLR: PLS predicts Y from X Combine PCA & MLR. PLS extracts factors from X in order to predict Y What is PLS Regression ?

When to use PLS ? To analyze two data tables describing the same I observations with J predictors and K dependent variables 1 … j … J 1...i...I1...i...I x i,j …... ……... Independent Variables Observations 1 … k … K 1...i...I1...i...I y i,k ……... Dependent Variables

General principle of PLS: 1 … j … J 1...i...I1...i...I x ij …... ……... Predictors X Observations t 1 … t ℓ... t L 1...i...I1...i...I t i,ℓ …... ……... Latent Variables t ℓ = Xw ℓ 1 … k … K 1...i...I1...i...I y i,k ……... Dependent Variables Predict NIPALS ℓ = t ℓ c T

PLS: Maps of the observations …... x ij t i,ℓ t 1 … t ℓ... t L …... ……... Latent Variables 1 … j … J 1...i...I1...i...I ……... X 1 … k … K y i,k ……... t ℓ = Xw ℓ NIPALS ℓ = t ℓ c T lv 2 lv 1 Observations: t ℓ I i

PLS: Maps of the variables …... x ij t i,ℓ t 1 … t ℓ... t L …... ……... Latent Variables 1 … j … J 1...i...I1...i...I ……... X 1 … k … K y i,k ……... t ℓ = Xw ℓ NIPALS ℓ = t ℓ c T lv 1 lv 2 Circle of correlations lv 2 lv 1 Common map w ℓ & c ℓ x x y x y y y

PLS: Predicting Y from X …... x ij t i,ℓ t 1 … t ℓ... t L …... ……... Latent Variables 1 … j … J 1...i...I1...i...I ……... X 1 … k … K y i,k ……... t ℓ = Xw ℓ NIPALS ℓ = t ℓ c T t ℓ = Xw ℓ & = t ℓ c T = XB pls Some Magic Here!

PLS: How do we explain Y from X? RESS =  (data – prediction) 2 Compare Data (Y) with Prediction (Yhat) RESS (REsidual Sum of Squares) 1 … k … K Y 1...i...I1...i...I ℓ = XB pls 1...i...I1...i...I

1 … k … K (-1) = X (-1) B pls 2...i...I2...i...I PLS: How do we predict Y from X? How well will we do with NEW data? Cross-validation. Here Jackknife 1 … k … K Y 1...i...I1...i...I Predict y 1 from X (-1) 1 … k … K Y (-1) 12...i...I12...i...I Predict y 2 from X (-2) … etc … Predict y I from X (-I)

PLS: How do we predict Y from X? How well will we do with NEW data? Cross-validation. Here Jackknife PRESS =  (data – jackknifed prediction) 2 Compare Data (Y) with Jackknifed Prediction (Y jack ) PRESS (Predicted REsidual Sum of Squares) 1 … k … K Y 1...i...I1...i...I jack = XB pls 1...i...I1...i...I

PLS Big Question: How Many Latent Variables? Compare RESS and PRESS, or use PRESS. Quick and Dirty: Min(PRESS) => Optimum number of Latent Variables

Back to cocoa Goals: Explain and Predict Sensory (Y) from Physico-Chemical (X)

X

Y

Correlation within the X set

Correlation within the Y set

Correlation between X and Y

Show The t (latent) variables

Show w

Show c

B pls: X to Y (in Z-scores)

B * pls from X to Y (original units)

Show RESS & PRESS < min PRESS for 4 Keep 4 latent variables

Plot w & t (1 vs 2)

Plot w & c (1 vs 2)

Show the circle of correlation

Conclusion Useful References (contain bibliography): Abdi (2007, 2003) see