1. Preprocessing With focus on NIR
Åsmund Rinnan , Frans van den Berg, Rasmus Bro, Søren Balling Engelsen, Lars Nørgaard , Jonas Thygesen

2. Introduction How often is it used? When is it used?
Techniques
Summary
How often is it used?
When is it used?
What methods are used?
Why is it used?
NIR is a measurement technique “screaming” for preprocessing

3. Introduction NIR Fructose Glucose 50 75 100 50 25 Introduction
Techniques
Summary
Fructose
Glucose 50 75
100 50 25

4. Introduction Techniques
Summary
Correction of light scatter
MSC/ ISC
PMSC
EMSC/ EISC
SNV
Derivation
Norris&Williams
Savitsky&Golay
Baseline-correction
Normalization
Detrend
Use of reference values
O-PLS
OSC OS
SIS

5. Introduction Why Baseline with a slope/ curve Nonlinearity
Techniques
Summary
Baseline with a slope/ curve
Nonlinearity
A “fat” baseline

6. ISC MSC Offset Normalization Raw spectrum Reference Techniques
Introduction
Techniques
Summary
Offset
Normalization
Raw spectrum
Reference
P Geladi, D MacDougal, H Martens (1985): Linearization and scatter correction for near-infrared reflectance spectra of meat, Applied Spectroscopy, 39,

7. MSC/ ISC
Introduction
Techniques
Summary
Offset
Normalization

8. EISC Offset Baseline Normalization Techniques Introduction Summary
H. Martens and E. Stark (1991): Extended multiplicative signal correction and spectral interference subtraction: new preprocessing methods for near infrared spectroscopy, Journal of Pharmaceutical and Biomedicinal Analysis, 9,
H Martens, JP Nielsen, SB Engelsen (2003): Light scattering and light absorbance separated by extended multiplicative signal correction. The application to near-infrared transmission analysis of powder mixtures, Analytical chemistry, 75,
DK Pedersen, H Martens, JP Nielsen, SB Engelsen (2002): Near-Infrared Absorption and Scattering Separated by Extended Inverse Signal Correction (EISC): Analysis of Near-Infrared Transmittance Spectra of Single Wheat, Applied Spectroscopy, 56,
Thennadil, SN, Martin, EB (2005): Empirical preprocessing methods and their impact on NIR calibrations - a simulation study, Journal of Chemometrics, 19, 77-89

9. EMSC (w/ 2. degree & wavelength)
Introduction
Techniques
Summary
Offset
Baseline
Normalization

10. PxSC Offset Baseline Normalization X = EM EI M I Techniques
Introduction
Techniques
Summary
Offset
Baseline
Normalization
X = EM EI M I
Isaksson, T.; Kowalski, B. R. (1993): Piece-wise multiplicative scatter correction applied to Near-Infrared diffuse transmittance data from meat products, Applied Spectroscopy, 47,

11. PMSC (window size = 35) Offset Normalization Techniques Introduction
Summary
Offset
Normalization

12. SNV Offset Normalization Techniques Introduction Summary
R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied Spectroscopy, 43,

13. SNV vs MSC
Introduction
Techniques
Summary
MSC
SNV

14. Detrend Offset Baseline Techniques Introduction Summary
R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989): Standard Normal Variate Transformation and De-trending of Near-Infrared Diffuse Reflectance Spectra, Applied Spectroscopy, 43,

15. Derivation Savitsky-Golay
Introduction
Techniques
Summary
Offset
Baseline
A Savitsky, MJE Golay (1964): Smoothing and differentiation of data by simplified least squares procedures, Analytical Chemistry, 36 (8),
P.A. Gorry (1990): General Least-Squares smoothing and differentiation by the convolution (Savitsky-Golay) method, Analytical Chemistry, 62,

16. Derivation Savitsky-Golay
Introduction
Techniques
Summary
Offset
Baseline

17. Derivation Savitsky-Golay
Introduction
Techniques
Summary
Offset
Baseline

18. Derivation Norris-Williams
Introduction
Techniques
Summary
Offset
Baseline
Normalization
Norris, KH (1983): in Food Research and Data Analysis (Eds: Martens H and Russwurm H Jr), Applied Science, London,
Norris KH, Williams PC (1984): Optimization of mathematical treatments of raw near-infrared signal in the measurement of protein in hard red spring wheat: I. Influence of particle size, Cereal Chemistry, 61,

19. Derivation Norris-Williams
Introduction
Techniques
Summary
Offset
Baseline
Normalization

20. Derivation Norris-Williams
Introduction
Techniques
Summary
Offset
Baseline
Normalization

21. Preprocessing PMSC: Sensitivite to window size
Introduction
Techniques
Summary
PMSC: Sensitivite to window size
EMSC: Small effect on baseline  Detrend is better
MSC and SNV are practical the same
Savitsky-Golay is more robust than Norris-Williams

22. Preprocessing Offset Baselinje Normalization (P)MSC/ISC/SNV x
Introduction
Techniques
Summary
Offset
Baselinje
Normalization
(P)MSC/ISC/SNV x
(P)EMSC/EISC
Detrend
Deriveration

23. Thanks for the attention