Measurement: Assessment and Metrics Presented by Dr. Joan Burtner Certified Quality Engineer Associate Professor of Industrial Engineering and Industrial Management

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 2 Overview Process Measurement as a Management Function Process Measurement as a Management Function Project Management Metrics Project Management Metrics Human Aspects of Data Gathering Human Aspects of Data Gathering Statistical Analysis Statistical Analysis Theory of Variation Theory of Variation Process Capability Process Capability Acceptance Sampling Acceptance Sampling

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 3 Process Measurement as a Management Function “Effective management of an organization depends on defining, gathering, and analyzing information that provides feedback on current performance as well as projecting future needs.” ETM627 Course Text p. 416 “Effective management of an organization depends on defining, gathering, and analyzing information that provides feedback on current performance as well as projecting future needs.” ETM627 Course Text p. 416 “Analysis refers to extracting larger meaning from data and information to support evaluation, decision making, and improvement.” Baldrige National Quality Program 2005 “Analysis refers to extracting larger meaning from data and information to support evaluation, decision making, and improvement.” Baldrige National Quality Program 2005 “Statistics is the science of turning data into information” ETM627 Course Text p. 417 “Statistics is the science of turning data into information” ETM627 Course Text p. 417

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 4 Typical Project Management Metrics Schedules met Schedules met Resources used Resources used Costs versus budget Costs versus budget Project objectives met Project objectives met Risks identified and eliminated or mitigated Risks identified and eliminated or mitigated Earned value analysis (planned vs. actuals) Earned value analysis (planned vs. actuals) Customer satisfaction Customer satisfaction

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 5 Human Aspects of Data Gathering Perception that excessive data collection and development of multiple metrics is: Perception that excessive data collection and development of multiple metrics is:  A reflection of management’s obsession with numbers  Not necessarily helpful in producing a better product or service Perception that organization is more interested in data collection than task performance Perception that organization is more interested in data collection than task performance Lack of understanding of the connection between what workers produce and the metrics by which management assesses performance Lack of understanding of the connection between what workers produce and the metrics by which management assesses performance

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 6 Statistical Analysis Central Tendency Central Tendency  Mean  Median  Mode Variation or Spread Variation or Spread  Range  Standard Deviation  Variance

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 7 Probability Distributions Widely-used distributions Widely-used distributions  Normal  Exponential  Weibull  Poisson  Binomial  Negative Binomial  Hypergeometric Graphs, Functions, and Applications Graphs, Functions, and Applications  See page 429 of course text

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 8 Advanced Statistical Methods for Managers Basic Hypothesis Testing Basic Hypothesis Testing  One Sample t or Z Tests  Two Sample t or Z Tests Advanced Hypothesis Testing Advanced Hypothesis Testing  Design of Experiments (ANOVAs)  Regression (Simple, Multiple, Non-linear) Visualization Visualization  Response Surface  Evolutionary Operation (EVOP) Incremental search for more optimal points on a response surface Incremental search for more optimal points on a response surface

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 9 Theory of Variation Common Cause Common Cause  Stable and predictable causes of variation  Inherent in all processes  Managers, not workers, are responsible for common cause variation Special Cause Special Cause  Unexpected or abnormal causes of variation  May result in sudden or extreme departures from normal  May also result in gradual shifts (trends)

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 10 Control Chart Types Control Charts Control Charts  Variables – based on continuous data X bar and R (mean and range) X bar and R (mean and range)  Attributes - based on discrete data P (proportion) P (proportion) C (count) C (count) Example of R Chart: Example of R Chart:

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 11 Control Chart Calculations for Variables Charts Xbar and R Control Chart Constants Xbar and R Control Chart Constants Control Chart Calculations Control Chart Calculations nd 2 A 2 D 3 D

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 12 Process Capability Analysis conducted on processes that have been shown to be “in-control” Analysis conducted on processes that have been shown to be “in-control”  Only common cause variation in range  Only common cause variation in mean Two standard measures Two standard measures  Cp Compares variability of process to specifications Compares variability of process to specifications  Cpk Compares variability of process to specifications Compares variability of process to specifications Is the process sufficiently centered? Is the process sufficiently centered?

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 13 Process Capability Calculations Calculations Calculations Evaluation Evaluation  Cpk > 1.33 Definitely Capable  1.00 < Cpk < 1.33 Possibly Capable  Cpk  1.00 Not Capable

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 14 Acceptance Sampling Definition: Acceptance sampling is the process of sampling a batch of material to evaluate the level of nonconformance relative to a specified quality level. Definition: Acceptance sampling is the process of sampling a batch of material to evaluate the level of nonconformance relative to a specified quality level.  Incoming product  Product moved from one process to another Types of samples Types of samples  Random  Stratified Sampling Decision Process Figure 15.3 in course text Sampling Decision Process Figure 15.3 in course text  D number of defective items (not number of defects)  n sample size  C acceptance number

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 15 Acceptance Sampling Not Recommended Why not use sampling to collect data? (according to course text pp ) Why not use sampling to collect data? (according to course text pp )  Customer requires 100% inspection  Relatively small number of items or services allows for ‘economical’ 100% inspection  The inspection method is built into production so that no defectives can be shipped  Self-inspection by trained operators is sufficient for the nature of the product produced

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 16 Sampling Plans Standards Standards  ANSI/ASQC Z1.4 which replaces MIL-STD 105  ANSI/ASQC Z1.9 for variables Potential errors (uncertainty risk) Potential errors (uncertainty risk)  Producer’s Risk “the probability of not accepting a lot, the quality of which has a designated numerical value representing a level that is generally desirable”  Consumer’s Risk “the probability of accepting a lot, the quality of which has a designated numerical value representing a level that is seldom desirable”

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 17 References Course Text: Course Text:  Westcott, R.T., Ed. (2006). Certified Manager of Quality/Organizational Excellence Handbook (3 rd ed.). Milwaukee: ASQ Quality Press. Additional Sources Additional Sources  “Baldrige National Quality Award Criteria”  Christensen, E.H., Coombes-Betz, K.M., and Stein, M.S. (2006). The Certified Quality Process Analyst Handbook. Milwaukee: ASQ Quality Press.

ETM 627 Fall 2008Dr. Joan Burtner, Associate Professor of Industrial Engineering Slide 18 Contact Information US Mail: US Mail: Mercer University School of Engineering 1400 Coleman Avenue Macon, GA Phone: (478) Phone: (478)