1 4. Model constraints Quimiometria Teórica e Aplicada Instituto de Química - UNICAMP

2 Principal component analysis (PCA) In Hotelling’s (1933) approach, components have maximum variance. –X = TP T + E –Components are calculated successively. –Components are orthogonal: T T T = Diagonal; P T P = I In Pearson’s (1901) and Eckart & Young’s (1936) approach, components explain maximum amount of variance in the variables. –X = AB T + E –Components are calculated simultaneously. –Components have no orthogonality or unit-length constraints.

3 Constrained least squares Solve under the constraint that A is non-negative, unimodal, smooth etc. Some constraints are inactive, e.g. PCA under orthogonality. If constraints are active, A is no longer the least- squares solution.

4 Why use constraints? Obtain solutions that correspond to known chemistry, making the model more interpretable. –Concentrations can not be negative. Obtain models that are uniquely identified. –Remove rotational ambiguity. Avoid numerical problems such as local minima and swamps. –Constraints can help ALS find the correct solution

5 Example: curve resolution of HPLC data (1) HPLC analysis of three coeluting organophosphorus pesticides. Diode-array detector gives a spectrum at each time point: X (time  wavelength). Beer-Lambert law says X = CS T + E. Initial analysis shows that three analytes are present. Data is from Roma Tauler’s web-site Download it and try for yourself!

6 Example: curve resolution of HPLC data (2) Unconstrained solution % of X explained Calculation time: 0.43 seconds CS

7 Example: curve resolution of HPLC data (2) Non-negativity constraints % of X explained Calculation time: 16 seconds CS

8 Example: curve resolution of HPLC data (3) Unimodality & non-negativity constraints % of X explained Calculation time: 16 minutes CS

9 Comments Active constraints always reduce % fit, but can give a more interpretable model. E3E3 X3X3 C3C3 E2E2 X2X2 C2C2 It is possible to ‘stack’ two–way data from different experiments, e.g. =+ E1E1 X1X1 C1C1 S

10 What sort of constraints might be useful? Hard target: known spectrum, a r = s Non-negativity: concentrations, absorbances Monotonicity: kinetic profiles Unimodality: elution profiles, fluorescence excitations Other curve shapes: Gaussian peaks, symmetry Selectivity: pure variables Functional constraints: first-principle models Closure: [A] t + [B] t + [C] t = y Orthogonality: useful for separation of variances...plus many more...

11 Conclusions (1) It is possible to mix constraints within the same mode, i.e. loadings 1 and 3 are non-negative, loadings 2 are unimodel. Chemical knowledge can be included in your model by using constraints. Constraints can improve the model making it –closer to reality –easier to understand –more robust to extrapolation

12 Conclusions (2) Mixed constraints can be applied using column-wise estimation: 1. Subtract contribution from other components 2. Estimate component under desired constraint –Bro & Sidiropoulos (1998) have shown that this is equivalent to solving where  is the unconstrained solution,

13 ALS for Tucker3 Step 0: Initialise B, C & G Step 1: Estimate A: Step 4: Estimate G: Step 5: Check for convergence. If not, go to Step 1. Step 3: Estimate C in same way: Step 2: Estimate B in same way:

14 Example: UV-Vis monitoring of a chemical reaction (1) Two-step conversion reaction under pseudo-first-order kinetics: A + B  C  D + E UV-Vis spectrum ( nm) measured every 10 seconds for 45 minutes 30 normal batches measured: X (30  201  271) 9 disturbed batches: pH changes made during the reaction

15 Example: UV-Vis monitoring of a chemical reaction (2) 3-component PARAFAC model has problems! spectra are difficult to interpret  highly correlated 

16 Example: UV-Vis monitoring of a chemical reaction (3) External process information A D No compound interactions allowed: Lambert-Beer law First-order reaction kinetics are known: Pure spectra of reactant and product known:

17 Example: UV-Vis monitoring of a chemical reaction (4) Constrained Tucker3 (1,3,3) model B C G = wavelength time batch A + E REACTION KINETICS KNOWN SPECTRA LAMBERT- BEER LAW X X = AG (C  B) T + E

18 Example: UV-Vis monitoring of a chemical reaction (5) Constrained Tucker3 (1,3,3) model Core array: G = [g | 0 g | 0 0 g 133 ] * * * * * fixed to known spectrum fixed to 1 st -order kinetics Rate constants are found! k 1 = 0.27, k 2 = Spectrum of intermediate is found!

19 Conclusions (3) It is possible to build ‘hybrid’ or ‘grey’ models where some loadings are constrained and others are left free – see the extra material which follows! If you already have some information about your chemical process, then include it in your model Using constraints can really help to uncover new information about your data (e.g. find spectra, estimate rate constants, test models).

20 Extra material: Black vs white models ‘Black-box’ or ‘soft’ models are empirical models which aim to fit the data as well as possible e.g. PCA, neural networks ‘White’ or ‘hard’ models use known external knowledge of the process e.g. physicochemical model, mass-energy balances Difficult to interpret Good fit Easy to interpret Not always available Good fit ‘Grey’ or ‘hybrid’ models combine the two. +

21 Extra material: Grey models mix black and white models ++ Total variation Systematic variation due to known causes Systematic variation due to unknown causes Unsystem- atic variation RESIDUALSMODEL REACTION KINETICS MECHANISTIC MODEL KNOWN CONCENTRATIONS

22 Extra material: Grey model B C G + = wavelength time batch A B C G + E REACTION KINETICS KNOWN SPECTRA LAMBERT- BEER LAW X A

23 Extra material: Grey model parameter estimation

24 Extra material: Grey model parameters White components Black components describe known effects can be interpreted 99.8% fit (corresponds well with estimated level of spectral noise of  0.13%) * * * * *

25 Extra material: Grey model residuals

26 Extra material: Off-line monitoring D-statistic Q-statistic (within model variation) (residual variation)

27 Extra material: On-line monitoring of disturbed batch