1. Applying Change Point Detection in Vaccine Manufacturing Hesham Fahmy, Ph.D. Merck & Co., Inc. West Point, PA 19486 hesham_fahmy@merck.com Midwest Biopharmaceutical Statistics Workshop (MBSW) - MAY 23 - 25, 2011

2. 2 Outline Definitions Detection methods CUSUM and EWMA estimators Case studies CUSUM and EWMA SSE Conclusions

3. 3 Methods of Detection Visual (Simple but Subjective) Raw data; run chart CUSUM chart EWMA chart Analytical (Complicated but Objective) Change-Point estimators; i.e. CUSUM, EWMA Mathematical Modeling; i.e. MLE, SSE

4. 4 Types of Variation  Common Causes – natural (random) variations that are part of a stable process Machine vibration Machine vibration Temperature, humidity, electrical current fluctuations Temperature, humidity, electrical current fluctuations Slight variation in raw materials Slight variation in raw materials  Special Causes – unnatural (non-random) variations that are not part of a stable process Batch of defective raw material Batch of defective raw material Faulty set-up Faulty set-up Human error Human error Incorrect recipe Incorrect recipe

5. 5 Cumulative Sum Control Chart CUSUM: cumulative sum of deviations from average A bit more difficult to set up More difficult to understand Very effective when subgroup size n=1 Very good for detecting small shifts Change-point detection capability Less sensitive to autocorrelation

6. 6 Exponentially Weighted Moving Average EWMA: weighted average of all observations A bit more difficult to set up Very good for detecting small shifts Change point detection capability Less sensitive to autocorrelation “EWMA gives more weight to more recent observations and less weight to old observations.” CUSUM Shewhart 0 1

7. 7 Process Shifts StepLinearNonlinearOthers

8. 8 Process Model in-control state out-of-control state t Change point, "Unknown"

9. 9 Change-Point Estimation Procedures CUSUM Change-Point Estimation Procedure (Page 1954) EWMA Change-Point Estimation Procedure (Nishina 1992) EWMA Change-Point Estimation Procedure (Nishina 1992)

10. 10 Example: Most Recent Reintialization at t =22 Actual Change-Point= 20

11. 11 Example: UCL=11 LCL=9 at t =19Most Recent time Actual Change-Point= 20

12. 12 Real Example

13. 13 CUSUM vs. EWMA

14. 14

15. 15 Simulated Case Studies To test different methods' (control charts/analytical) capability for identifying change-points Case # 1: small shifts/drifts Case # 2: mirror image of case # 1 Case # 3: large shifts/drifts

16. 16 Case #1

17. 17 Case # 1 1 & 6: In-control process; N(0,1) 2 : -ve linear trend; 0.1 sigma 3 : Step shift; N(-3,1) 4 : Step shift; N(-1.5,1) 5 : +ve linear trend; 0.1 sigma

18. 18 CUSUM (V-Mask) Detection Criterion: Slope Change Diagnostic Sequence Plot

19. 19 CUSUM; Exact Profile

20. 20 CUSUM – Target Adjusted

21. 21 EWMA; Detection Criterion: Slope Change

22. 22 EWMA; CUSUM Same Profile

23. 23 Case # 2 “Mirror Image” 1 & 6: In-control process; N(0,1) 2 : +ve linear trend; 0.1 sigma 3 : Step shift; N(3,1) 4 : Step shift; N(1.5,1) 5 : -ve linear trend; 0.1 sigma

24. 24 CUSUM Chart Mirror Image Case # 2 Case # 1

25. 25 CUSUM – Target Adjusted

26. 26 EWMA Chart;

27. 27 EWMA Chart;

28. 28 EWMA Chart;

29. 29 Case # 3 1 & 6: In-control process; N(0,1) 2 : +ve linear trend; 0.5 sigma 3 : Step shift; N(12.5,1) 4 : Step shift; N(9.5,1) 5 : -ve linear trend; 0.38 sigma

30. 30 CUSUM

31. 31 CUSUM – Target Adjusted

32. 32 EWMA;

33. 33 EWMA;

34. 34 EWMA;

35. 35 Detection by SSE Pick a window of about 30 points including the “investigated point” Fit a two-phase regression using all possible change-points & calculate the SSE Plot possible change-points vs. their SSEs

36. 36 Examples from Case # 2

37. 37 Examples from Case # 3

38. 38 Conclusions Change-point problem is general and can be applied in many applications such as 4 parameter logistic regression and degradation curves. Another application in manufacturing processes includes detection of the change-point for process variance. It is preferred to combine both analytical and visual techniques; in addition to process expertise; to get accurate results.

39. 39 References Fahmy, H.M. and Elsayed, E.A., Drift Time Detection and Adjustment Procedures for Processes Subject to Linear Trend. Int. J. Prod. Research, 2006, 3257–3278. Montgomery, D. C., Int. to Stat. Quality Control, 1997, (John Wiley: NY). Nishina, K., A comparison of control charts from the viewpoint of change-point estimation. Qual. Reliabil. Eng. Int., 1992, 8, 537–541. Pignatiello, J.J. Jr. and Samuel, T.R., Estimation of the change point of a normal process mean in SPC applications. J. Qual. Tech., 2001, 33, 82–95. Samuel, T.R., Pignatiello Jr., J.J. and Calvin, J.A., Identifying the time of a step change with X control charts. Qual. Eng., 1998, 10, 521– 527.

40. 40 Acknowledgements Lori Pfahler Julia O’Neill Jim Lucas