1. Department of Chemical Engineering National Tsing Hua University
Multivariate Statistical Monitoring And Fault Diagnosis Of Two-Dimensional Dynamic Batch Processes
Yuan YAO
Department of Chemical Engineering
National Tsing Hua University
26/10/2012

2. Multivariate Statistical Monitoring of Batch Processes
Part 1
12:28

3. Batch Process Definition Applications
A process that leads to the production of finite quantities of material by subjecting quantities of materials to an ordered set of processing activities over a finite period of time using one or more pieces of equipment (American National Standard)
Applications
Pharmaceuticals, polymers, biochemicals, food products and specialty chemicals…
12:28

4. Why Batch Process Monitoring
Requirement from global competition
Consistent and high quality
Operation safety
Environmental guidelines
Minimal energy and raw materials consumption
To achieve this, process performance must be monitored in real-time
12:28

5. Multivariate Statistical Process Monitoring
MSPC
Model
Normal
Operation
Data
Database
Current Batch Data
Process Data
Temp, Stroke, Velocity, Pressure, …
Feedback Control
and Optimization
Online Monitoring and Fault Diagnosis
12:28

6. Multivariate Statistical Projection Techniques
Transformation of process variables into latent variables
Orthogonality
Dimension reduction
Extraction of process information and knowledge
The basic tools
Principal component analysis (PCA)
Partial least squares (PLS)
12:28

7. PCA and PLS PCA PLS Coordinates transformation
Dealing with single data set (I or O)
Extraction of main variances
PLS
Emphasize the covariance
between two data sets (I & O)
PCA can be regarded as a kind of coordinates transformation.
The significances of PCA is it transform original data set to a set of orthogonal principal components, and use these principal components, the dimensions of data are greatly reduced.
This is an example of PCA. In this figure, there are two variables that positive correlate to each other. However, because of the existence of noise, they are looked not exactly correlated. We transform the coordinates into this one, and call the value of data in the new coordinate principal components. We find the first principal components contain most variance information and the second ones contain noise. So we only keep the first ones to represent the whole original data set, thus, the dimensions are reduced from two dimensions to one dimension.
PCA are used to deal with single data set and capture correlations between variables in this data set.
Some times, we concern more about the correlations between two data sets, and want to explain these correlations in lowest dimensions. In this situation, PLS can do this job. X T U Y
12:28

8. PCA Decomposition
12:28

9. PCA-Based Monitoring and Diagnosis
Multivariate Statistics
SPE:
Independence of residuals T2
Multivariate normal distribution of scores
Typical monitoring and diagnosis charts
PLS-based monitoring can be conducted in similar way
Original data space X
Score space
TPT
Residual space E + T2
SPE
Kourti (2006)
12:28

10. PCA Score Plot
Kourti et al. (1996)
Hodouin et al. (1993)
12:28

11. PCA loading Plot
Lu et al. (2004)
12:28

12. Normalization Removing means and equalizing variances Benefits
Eliminating the effects of variable units and measuring ranges
Emphasize correlations among variables
12:28

13. An Example of Batch Process: Injection Molding
A cycle
Plastication
Packing - Holding
Mold Close
Filling
Mold Open
Cooling M
12:28

14. Batch Process Data Matrix
12:28

15. Batch-wise Unfolding and NormalizationK
Batches
Variables
Time 1 I J 1 J I JK T1 T2 T3
Nomikos and MacGregor (1994, 1995)
12:28

16. Time-wise Unfolding and NormalizationJ B1 1 K IK B3 B2
Batches
Variables
Time 1 I J K
Wold et al. (1998)
12:28

17. Data From Penicillin Fermentation
Variable 1
Variable 3
Variable 2
Raw
Batch-wise Normalized
Time-wise Normalized
12:28

18. After Batch-wise Normalization
Raw Data
Normalized Data
Raw Data
Normalized Data
12:28

19. After Time-wise Normalization
Raw Data
Normalized Data
Raw Data
Normalized Data
12:28

20. Multiway PCA (MPCA) Properties
PC score vectors contain information on batch-to-batch variation
Loading matrices reflect variable behaviour over time t1 t2 tK
v1, v2, v3, …vJ b1 b2 b3 bI
Score vectors
Loading matrices
Nomikos and MacGregor (1994, 1995)
12:28

21. Online Monitoring Based on MPCA
Nomikos and MacGregor (1994)
12:28

22. Features of MPCA Focus on between-batch variations
Nonlinear and dynamic components are reduced or eliminated
Batch duration is required to be equalised
Future measurements need to be estimated
12:28

23. Features of Time-wise MPCA
Focus on through-batch behaviour
Non-linear and dynamic components still exist in the data
Future measurements estimation is not needed
Batches can be of different durations
12:28

24. Two-Stage Approach Stage 1: Batch-wise Stage 2: rearrangement
Unfolding and
normalization
Time, K
Batches, I t1 t2 tK
Variables, J
v1, v2, …vJ
v1, v2, …vJ
v1, v2, …vJ b1 b2 b3 bI
v1, v2, …vJ t1 t2 b1 t3 tK t1 t2
Rearrangement b2 t3 tK t1 t2 bI t3 tK
12:28

25. Features of The Two-Stage Approach
No estimation of future observations
No need to equalize batch durations
Major nonlinear and dynamic components are reduced or eliminated
12:28

26. Future Data Estimation in Online Monitoring
Different estimation methods
Zero deviations
Assume future measurements to operate along the mean trajectory
Current deviations
Assume future measurements to continue at the same level as present time
Missing data
Fill the future measurements based on variable correlations
Current time
NO future data
Current
batch
P(1xJK) t
12:28

27. Batch Trajectory Synchronization
Methods
Cutting to minimum length
Missing data
Indicator variables
Dynamic time warping (DTW)
Estimation of batch progress
Rothwell et al. (1998)
Kourti (2003)
Nomikos and MacGregor (1995)
Kassidas et al. (1998)
Undey et al. (2003)
12:28

28. Dynamic PCA (DPCA) Conduct PCA on an expanded data matrix 12:28
Ku et al. (1995)
12:28

29. Batch Dynamic PCA (BDPCA)
Chen and Liu (2003)
12:28

30. Multiphase Batch Processes
Motivations
Multiphase is an inherent nature of many batch processes
Each phase has its own underlying characteristics
Process can exhibit significantly different behaviors over different operation phases
To develop a phase-based model to reflect the inherent process stage nature can improve process understanding and monitoring efficiency
12:28

31. Sub-PCA Recognition of batch processes: Three steps:
A batch process can be divided into phase reflected by its changing process correlation nature
Despite that the process may be time varying, the correlation of its variable will be largely similar within the same phase
Three steps:
Phase division in terms of process correlation
Sub-PCA modeling
Online monitoring
Lu et al. (2004)
12:28

32. Phase Division and Process Modeling With Sub-PCA
12:28

33. Phase Division Results of Injection Molding Process
Mold Close
Filling
Packing-holding
Cooling
(Plastication)
Mold Open
V/P Transfer
Gate Freeze
12:28

34. Extensions of Sub-PCA Phase-based monitoring of uneven-length batches
Phase-based quality prediction and control
Phase-based monitoring with transition information …
12:28

35. Other Research Efforts
Nonlinearity
Multiscale …
12:28

36. Two-Dimensional Dynamic Batch Process Monitoring
Part 2
12:28

37. Batch process dynamics monitoring
Within batch dynamics
Batch-to-batch dynamics
Slow response variables, variations in materials, batch-wise control law, …
Existing methods
PCA/PLS + time series model (Wold, 1994)
Dynamic PCA (DPCA) (Ku et al., 1995)
Batch Dynamic PCA/PLS (Chen & Liu, 2002)
MPCA/MPLS + prior batch information
(J.Flores-Cerrillo & MacGregor,2004)
The long term dynamics are considered by including some prior batch information
Short term
Long term
batch1
batch2
batch3
There are totally two kinds of dynamics in batch processes. …
Several methods are proposed to model dynamics.
PCA/PLS + time series model and Dynamic PCA model are used to model dynamics in continuous processes.
In 2002, DPCA is extended to batch process, and used to model short term dynamics with a single batch.
In 2004, J.Flores-Cerrillo & MacGregor include prior batch information in current batch’s MPCA and MPLS model in order to model two kinds of dynamics simultaneously. Their method models a long term dynamics within several batches. The dynamics are still modeled in a single direction that is time direction.
If we re-organize the figure like this, we can find batch-to-batch dynamics can be considered as a kind of dynamics in different direction with with-in batch dynamics. Thus, we can form a two-dimensional model to capture both directions’ dynamics.
This model is more accurate than conventional method, and thus can provide more accurate monitoring.
12:28

38. Two-dimensional dynamic PCA (2D-DPCA)
Score space monitoring based on 2D-DPCA
Multiphase 2D batch process monitoring
Subspace identification for 2D dynamic batch process statistical monitoring
Non-Gaussianity
12:28

39. Research motivations
Features of 2D dynamic batch data
12:28

40. Two-dimensional dynamic PCA (2D-DPCA)
A fact
Cross- and 2D autocorrelations (dynamics) information is reflected by the correlations among current measurements and lagged variables in region of support (ROS)
Idea
Data matrix augmented by including all lagged variables in ROS
Performing PCA to extract correlation information (cross-correlation and 2D dynamics)
Monitoring based on SPE
Advantages
Better monitoring results with more dynamic information built in the model
No prediction of future measurement is needed …
I will talk about the ROS determination problem more detailed in the work progress part.
12:28

41. Key points of the 2D-DPCA algorithm
Define data matrix as below
Perform PCA on this augmented data matrix
Extract simultaneously 2D auto-correlated and cross-correlated relationships
Monitoring: SPE
Residual space contains only noise
Satisfy statistical assumption of independence
Control limit of SPE can be estimated by Χ2 distribution
12:28

42. ROS determination problem
A subset of lagged variables which can reflect process dynamics (autocorrelations and lagged cross-correlations) correctly
Difficulty in ROS determination
Variety of reasons causing batch process dynamics
No uniform shape or order for all batch processes
Property of the lagged variables in ROS
Reasonable predictor variables which can be regressed to variables’ current values
Key idea of ROS determination
Similar to the variable selection problem in regression model building
12:28

43. ROS autodetermination based on backward eliminations
Initial ROS selection
Lagged variables with significant correlations to current samples
Simple regression method (Gauchi, 1995; Gauchi and Chagnon 2001)
Student t-test for the slope coefficient
Target proper ROS determination
Iterative stepwise elimination (ISE) (Boggia et al., 1997)
In each elimination cycle, the independent variable with the minimum importance is eliminated
An index (e.g., PRESS, AIC) is used as a criterion to evaluate the regression models built in iterations
The best choice of the ROS the candidate region corresponding to the best model
12:28

44. Procedure of 2D-DPCA with autodetermined ROS based on backward eliminations
12:28

45. Stepwise regression Major procedure Advantages
The predictor variable with the largest correlation with the criterion variable enters the equation first, if it can pass the entry requirement based on statistic significance
The other variable is selected based on the highest partial correlation, if it can pass the entry requirement
The variables already in the equation are examined for removal according to the removal criterion
The last two steps are run iteratively
Variable selection ends when no more variables meet entry and removal criteria
Advantages
More robust than backward elimination
Get rid of the redundant information provided by the candidate variables and noises
12:28

46. ROS autodetermination based on forward iterative stepwise regressions
12:28

47. Case study I
Batch process model
Residuals of 2D-DPCA model
12:28

48. Faults to detect Fault 1 Fault 2
Correlation structure change of x2 from batch 61
Fault 2
A small process drift on variable x2 from batch 61
Adding a signal that increases slowly with time and batch
12:28

49. Monitoring of correlation structure change
2D-DPCA
MPCA with prior batch information
12:28

50. Monitoring of small process drift
2D-DPCA
MPCA with prior batch information
12:28

51. Two-dimensional dynamic PCA (2D-DPCA)
Score space monitoring based on 2D-DPCA
Multiphase 2D batch process monitoring
Subspace identification for 2D dynamic batch process statistical monitoring
Non-Gaussianity
12:28

52. Score information in dynamic batch processes
Discussions on SPE and T2
Concern about different kinds of information
Complementarities of each other
Risk of not using score information in monitoring
Possible missing alarm
Scores from dynamic (including 2D dynamic) PCA model
Not satisfy the statistical assumption for control limits calculation
Autocorrelations
Lagged cross-correlations
Reasonable T2 control limits can not be achieved unless the dynamics in score values are filtered
2D multivariate autoregressive (AR) score filters
Extracting score dynamics with AR filters
Calculating T2 with filtered scores
Previously, I have introduced the monitoring based on 2-D-DPCA model. It makes use of statistic SPE but not T2. These two statistics are used for different purposes. ….
12:28

53. 2D multivariate AR score filters
Requirements in filter design 2D
Multivariate
Suited to different dynamic structure in score space
No existing filter can be directly used
Filter design
Score ROS autodetermination
Filter calculation
12:28

54. 2D-DPCA based modeling and monitoring in both score and residual SPE
12:28

55. Case study II Batch process model Faults to detect
Fault 1: a correlation structure change in x2 starting from batch 61
Fault 2: a drift starting from batch 61
Fault 3: a drift in the trajectory of x2 only in batch 61
Only affecting SPE in batch 61
Affecting T2 in batch 61 and the following batches
12:28

56. Dynamics in scores Lagged cross-correlations
PC1
Lagged cross-correlations
between scores in time directions
PC2
PC3
PC4
PC1
PC2
PC3
PC4
12:28

57. Dynamics in filtered scores
PC1
Lagged cross-correlations
between filtered scores in time
directions
PC2
PC3
PC4
PC1
PC2
PC3
PC4
12:28

58. Monitoring results of fault 1
Unfiltered scores
Filtered scores
12:28

59. Monitoring results of fault 2
Unfiltered scores
Filtered scores
12:28

60. Monitoring results of fault 3
SPE T2
12:28

61. Two-dimensional dynamic PCA (2D-DPCA)
Score space monitoring based on 2D-DPCA
Multiphase 2D batch process monitoring
Subspace identification for 2D dynamic batch process statistical monitoring
Non-Gaussianity
12:28

62. Multiphase 2D dynamic batch processes
Characteristics
The structures of variable cross-correlations and 2D dynamics may change from phase to phase
Not proper to build a single 2D-DPCA model for the whole batch operation
Difficulties in phase division and modeling
To take 2D dynamics into consideration in phase division, the correlations between current measurements and lagged measurements in ROS need to be extracted, which means ROS determination is necessary before phase division
Without phase division, different phase ROS cannot be determined
12:28

63. Major steps of multiphase 2D-DPCA modeling
Iterative phase division
Major steps of multiphase
2D-DPCA modeling
12:28

64. Case study III A two-phase 2D dynamic batch process Fault to detect
Phase I
Phase 2
Fault to detect
A small drift in the trajectories of x2 starting from batch 81
12:28

65. Monitoring results
Multiphase 2D-DPCA
Regular 2D-DPCA
12:28

66. Two-dimensional dynamic PCA (2D-DPCA)
Score space monitoring based on 2D-DPCA
Multiphase 2D batch process monitoring
Subspace identification for 2D dynamic batch process statistical monitoring
Non-Gaussianity
12:28

67. Motivations Shortcoming of lagged variable model structure Solutions
Too many variables which make the contribution plots messy and hard to read
Solutions
Reducing the number of variables
Summarizing the process dynamics information with a small number of state variables (subspace identification (SI)) instead of the lagged variables in ROS
Extracting variable correlations and dynamics for process monitoring
Building 2D-DPCA model using the state variables and the current variables
12:28

68. SI-2D-DPCA Canonical variate analysis (CVA) for SI SI-based 2D-DPCA
An application of canonical correlation analysis (CCA) on SI
Extracting the relationship between past (lagged measurements in ROS) and future (current measurements)
Finding the sets of orthogonal canonical latent variables in both data spaces which are most correlated
Achieving good prediction accuracy using fewest latent variables
SI-based 2D-DPCA
Applying 2D-DPCA on an expanded matrix
Y: matrix of process data
X: matrix of states
12:28

69. Case study V Batch process model Fault
Variable correlation structure change from the start of batch 61
12:28

70. Subspace identification
CVA
Canonical correlations
r: a vector indicating canonical correlations
CCA
5.3462e-007
3.6441e-007
5.1788e-008
1.955e-008
1.955e-008
12:28

71. Monitoring results 2D-DPCA using states variables
2D-DPCA using lagged variables
12:28

72. Fault diagnosis results
2D-DPCA using states variables
2D-DPCA using lagged variables
12:28

73. Two-dimensional dynamic PCA (2D-DPCA)
Score space monitoring based on 2D-DPCA
Multiphase 2D batch process monitoring
Multi-time-scale dynamic PCA
Subspace identification for 2D dynamic batch process statistical monitoring
Non-Gaussianity
12:28

74. Handling non-Gaussianity
Non-Gaussianity in batch process data
Dynamics Break the statistical
Multiple operation phases hypothesis of
… Gaussian distribution
Solutions
Score filter, phase division, … Or
Control limits estimation from non-Gaussian information
2D-DPCA + Gaussian mixture model (GMM)
12:28

75. Case study: penicillin fermentation
The fed-batch penicillin fermentation is a two-phase process, which starts with a batch preculture for biomass growth. In this phase, most of the initially added substrate is consumed by the microorganisms and the carbon source (glucose) is depleted. When the glucose concentration reaches a threshold value, the process switches to the fed-batch phase with continuous substrate feed.
A two-phase benchmark batch process
Phase I: batch preculture for biomass growth
Phase II: fed-batch phase with continuous substrate feed
Batch-to-batch dynamics
Disturbances in substrate feed rate vary from batch to batch in a correlated manner
12:28

76. Monitoring results of a temperature controller fault
Conventional 2D-DPCA
To simulate this fault, the temperature setpoint is changed from 298 K to K.
12:28

77. Monitoring results of a temperature controller fault (Cont.)
Non-Gaussian 2D-DPCA
12:28

78. Conclusions A first study of batch process dynamics in two dimensions
A first combination of PCA technique and 2D model structure
ROS autodetermination
2D multivariate AR score filters for score space monitoring
An iterative procedure to determine the phase division points and the ROS for each phase
Combination of SI technique and 2D-DPCA model structure for clearer fault diagnosis
Handling non-Gaussianity
12:28

79. Prospects Trajectory alignment on 2D dynamic batch process data
2D batch process data filtering
Multiple sampling rate
Quality prediction for 2D batch processes
Non-stationary 2D dynamics
Different scales of batch dynamics in two directions
12:28

80. Thank you!
12:28