Module 12 – Measuring Success LeanSigma® Facilitator Training Module 12 – Measuring Success

Topics Measuring Success Process performance measures……………………………………… 5 - 17 Process capability ………………………………………………………… 18 - 23 Program performance measures ………………………………… 24 - 34

Lean Sigma has six steps to optimize processes. Lean has a methodology (similar to DMAIC for Six Sigma). You’ll will use this methodology each time you run a kaizen or lean project.

Introduction We’ll look at several types of measures: Process performance: cycle time, KPIs, DPMO and yield Process capability: capability indices Program performance: pace, cost savings, implementation progress Selected process measures should follow from CTQs We want quantitative measures of a process because they’re much more objective than qualitative or ad hoc data. Performance data can help us to set a baseline, so we know when to intervene with more process improvement work. Establishing Baseline Performance At this point in the improvement process you should know what you want to accomplish (project Charter), what to measure (CTQs Operational Definitions), and that the measurement system has been validated (Measurement System Analysis). The next step is to establish a baseline to tell you where you are today. From that baseline you can quantify the gap between current performance and required performance, and develop action plans to close the gap. The baseline performance measurement should be documented as part of the Six Sigma project - a reference point to compare improvements. Baseline from the goals defined Use the baseline to quantify the gap between actual and desired performance & hence action plans Document the baseline Baselining using yield or Sigma level measures defects or defectives received by the customer (i.e. output measures of our processes) and then drives performance upstream Baselining of a continuous response of a process will be covered in SPC & Capability

Cycle time is our most common measure of process improvement. We create an estimate of future state cycle time in kaizens based on the improvements we plan to make. After the future state is implemented, you’ll measure the actual cycle time improvement. This validates how much time we’ve redeployed to other work. You’ll capture this measure in the Lean Sigma PMO Annualized Labor Hours Saved Cycle time is generally not measured on an on-going basis. You will receive a demo of the PMO within the next few weeks.

Other common process improvement measures Reduction in lead time May support a reduction in DAP; but often process improvements must first be implemented in other, related processes Process improvement measures are often not measured on an on-going basis, but rather before and after process changes (2x) Typical kaizen event measures

On-time delivery, query responsiveness, defect rate, throughput, etc. Key performance indicators (KPIs) are tracked to monitor the health of the process over time. Define Collect Analyze Report Take Action Identify the “vital few” measures that tell the story of process performance - Define SMART Metrics: • Specific in that your metrics are specific and targeted to the area you are measuring • Measurable in that you can collect data that is accurate and complete • Actionable in that the metrics are easy-to-understand and interpret • Relevant simply means don't measure things that are not important • Timely metrics are those for which you can get the data when you need it - Establish a baseline to tell you where you are today. From that baseline you can quantify the gap between current performance and required performance, and develop action plans to close the gap. - Clarify who is responsible for the measurement system - Measures must be actionable in order to add value On-time delivery, query responsiveness, defect rate, throughput, etc.

Clear metrics definitions are important! Each KPI must have an Operational Definition: An exact definition of the measure How it should be measured Measurement frequency Who should measure The threshold or expected service level Reporting: KPIs are evaluated in 30 / 60 / 90 day post-kaizen reviews. KPIs are monitored on an on-going basis by Process Owners, who are accountable for taking the appropriate action when results fall below corresponding thresholds Clear, documented metrics definitions are needed: - To ensure the appropriate measure-based decisions are made - To facilitate consistent measures across different geographic locations. Collection: Automate, where possible KPI results may reflect process improvements, if available, both before and after process changes are incorporated

Defects per unit and per million opportunities are common process measures. Term Definition Defective Any transaction that contains at least 1 defect (error) Defect Any part of a single transaction that has an error Opportunity Any part of a single transaction that can cause a defect A transaction could have any number of opportunities contained within it, a measure of complexity Every chance to do something either “right” or “wrong” DPU Defects Per Unit – Calculates the percentage of the time in decimal form how many defects occurred out of the total transactions evaluated DPMO Defects Per Million Opportunities – Calculates the number of times out of a million a defect would occur based upon the total opportunities for error These measures will NOT be appropriate for all processes. These measures may be used to assess the impact of process changes/improvements, and not for on-going process monitoring.

Calculating DPMO A project team is working on improving invoice accuracy The number of defect opportunities per invoice is 10. The team evaluated 100 invoices and discovered 35 defects DPMO = (# of defects x 1,000,000) (# of Defect Opportunities/Unit) x # of Units DPMO = 35 x 1,000,000 10 x 100 “Six Sigma quality” means DPMO ≤ 3.4

Calculating DPMO A delivery process of a courier company has 3 defect opportunities (timeliness, accuracy of delivery, accuracy of invoice) The company has delivered 5000 mail pieces, 8 of them were too late, 3 of them delivered to the wrong address, and 2 had a wrong amount on the invoice The DPMO is

Process Yield Concepts LeanSigma Yield concepts are: Throughput Yield – one process step: ratio of defects vs. defect opportunities Rolled Throughput Yield (RTY) – multiple process steps: multiplication of Throughput Yields Normalized Yield – multiple process steps: root of the multiplied Throughput Yields (“average process yield”) “Traditional” Yield concepts: First Time Yield – ratio of defectives vs. units processed Final Yield – ratio of defectives (after rework) vs. units processed Yield measures may be used strictly to measure process improvement

FTY + p(d) x 100% = 100% First Time Yield FTY – the percentage of units that pass through an operation without any defects If you describe the yield as 80%, the true performance of the process is not measured FTY = 20%, therefore the percentage defective = 80% FTY + p(d) x 100% = 100% Percentage Defective

“Traditional” Yield Metrics First Time Yield (FTY) Final Yield (FY) Focus on Defectives and units No. of units: 1057 No. of defective: 58 FTY: 94.5% Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 No. of units: 1057 No. of defective: 45 FY: 95.7% Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Rework

FTY Example 1056 invoices were processed in one day. 924 passed and 132 failed for various reasons # of bad units + # of good units = Total # of units # of bad units can be expressed as a probability of a defective, p(d): or a percentage defective: p(d) x 100% = percent defective FTY + percent defective = 100% FTY = 87.5%, percent defective = 12.5%

LeanSigma Yield Metrics Throughput Yield (TY) Focus on Defects and defect opportunities No. of defect opportunities per unit: 4 No. of units processed: 268 No. of total defect opportunities (4x268): 1072 No. of defects: 34 Throughput Yield: 96.8% Step 1

Rolled Throughput Yield What is the probability of producing this product error free? Invoice Generation Invoice Processing Billing Output TY = 97% Op1 x TY = 98% Op2 x TY = 91% Op3 = 86.5% 86.5% probability an invoice will pass through defect-free

How can we establish thresholds for selected process measures? Thresholds or “specifications” should be established for all KPIs to flag the need for adjustment to the process Specifications are usually based on: Historical results Compliance/regulatory requirements Benchmark data Management judgment In most cases, IMS specs are one-sided limits defined by internal customers Coding Accuracy: Error (defect) rate < 1% Panel fulfillment > 90% Specifications are often documented in Service Level Agreements (SLAs)

How can we determine if our process measures can routinely comply with established specs? Process capability is the inherent ability of a process to meet the specifications The performance of the process when it is stable and predictable, or operating in a state of statistical control, i.e., the mean and the variation in the process are not shifting Assumes no “special cause” variation, e.g., a “market event”, or a hurricane and resulting production delays. Verify this! Process capability measures provide insight as to whether the process has: A centering issue relative to the specifications A variation issue A combination of centering and variation issues Inappropriate specifications Process capability measures provide a baseline assessment No special (or, “assignable”) cause variation: Can often be verified by generating and assessing plots of the data and by statistical tests for normality. Common Cause: Variation caused by unknown factors resulting in a steady but random distribution of output around the average of the data. It is the variation left over after Special Cause variation has been removed and typically (not always) follows a Normal Distribution.

When specifications cannot be changed, actions must be taken to improve process capability Incapable Process: - This scenario occurs often in the six sigma projects; it’s more difficult to resolve than a centering issue - In this scenario, the process is centered between the specifications at the mean; however, process variability is high - Resolution: Identify the variables that will reduce variability in the process outputs and take the necessary action to modify them - Example: Data Investigation (DI) TATs are inconsistent. Variables that impact DI TATs may include availability of the mainframe or UNIX environments, training of investigators, etc. Off-Target Process: - In this scenario, process variability is relatively low, but the process consistently shows too many defects - Resolution: Identify the variables that will shift the process over to the mean - If we cannot change the specs, this is usually the easiest option to improve process capability LSL = Lower Spec Limit USL = Upper Spec Limit Target is usually the midpoint between the specs

Standard deviation describes how far results in a process vary from the mean result. Processes with higher standard deviations have more variation. They are less predictable and less stable than healthy processes. Processes with less variation will have a higher “sigma level” and fewer defects. Variation (Dispersion) Normal Distribution (“Bell”) Curve The bell curve is defined / characterized by the mean and standard deviation Standard Deviation =

We assess process capability using process capability indices. Oct 2010 We assess process capability using process capability indices. Cp = = What is possible if your process is perfectly centered The process potential Cpk = minimum ( , ) The reality of your process performance How the process is actually running relative to the specifications The “k” is the penalty for being off-center specification width process width (USL – LSL) 6s (USL – X) 3s (X - LSL) 3s (c) IMS Health Confidential Six Sigma Green Belt Certification

Oct 2010 The “sigma level” of the process provides another measure of process capability. σlevel = minimum ( , ) Number of standard deviations between the center of the process and the nearest specification “Six-sigma quality” means: Cp ≥ 2.0 Cpk ≥ 1.5 σlevel ≥ 4.5 (USL – X) s (X - LSL) s Sigma level at 4.5 allows for process drift Process capability should be periodically reassessed, especially following the implementation of process improvements. (c) IMS Health Confidential Six Sigma Green Belt Certification 23

Program metrics help us monitor the health and benefits of the LeanSigma program. Are we achieving our mission to deliver more value to our customers and stakeholders by identifying and implementing measureable process improvements? Regular Lean governance meetings help us: Align on priorities and targets Validate results meet expectations Ensure support Mitigate risks Costs: Lean facilitator & SME time Travel $ Implementation costs Benefits: Enable IMS to deliver on strategic priorities Transform the culture LeanSigma Mission: Pursue process opportunities that directly enable IMS to deliver on strategic priorities (transformation, revenue, EBIDTA, and sales excellence). Transform the culture so employees see opportunities for continuous improvement in everything they do. Governance meetings: With the lean council (Jim Berkshire, Paul Thomson, lean leads), with GTSO management, with Ari and his team.

Lean program reporting summarizes results from all Lean events. Cadence Pace of events by month, department & geography Benefits Net EBIDTA savings Annualized hours saved (capacity redeployed, FTE savings) Percent improvement cycle / lead time, delays, etc. Highlights of exceptional events, case studies Implementation Percent of identified savings removed from budget Percent of kaizen action items completed Focus areas Prioritization of processes based on exec sponsor discussions Pipeline of Lean events Cultural entrenchment Number of employees trained / certified Number of employees participating in kaizens Communications 95% of reporting is enabled by web queries against the Lean PMO

Annualized Labor Hours Saved Sample Dashboard U.S. Events by Month Process Focus to Date n= 110 completed events planned 2011 2012 80.23% of all U.S. kaizen action items completed 2012 U.S. EBIDTA Contribution: $817K *100% of savings realized Annualized Labor Hours Saved = 31,086

Cost Savings Detail Example

Qualitative Reporting Sample

Case Study: U.S. Market Research Client Support February 2011 Sponsor: Manny Triggiano Process Owner: Brenda Capobianco Lean Process Lead: Carissa Waltenbaugh Team: Kevin Downing, Chris Davies, Cathy Pileggi-Jefferson, Katrina Kulp, Kyle Burke, Michele Agnoli, Joy Cross, Gerard Avillo, Lisa Cooney, Jeff Northern Key Kaizen Findings 90 Days After the Kaizen Issues Noted Process Changes A pilot was completed to test the central support pool for smaller clients. The centralized model went live June 13, 2011. QMS was enforced as a central information system. Tier 1 (simple) and 2 (complex) resource pools were created. Checklists and decision trees were created to clarify info needs. Clients are assigned a CSA even if they call only a few times a year. Pool CSA resources so questions are answered by the first available CSA. Some CSAs less able to handle complex questions. Triage client questions and assign CSA resources to a Level 1 pool (simple) or Level 2 pool (complex). CSAs sometimes call multiple times to refine the client question. Create a checklist to guide CSAs when refining details of a client’s question. Create a decision tree to identify what research is required for a particular question. Improve how info is organized on ARC. CSAs sometimes do unnecessary research.

Case Study: U.S. Market Research Support Findings 15 months after implementation Results: The central service center remains in place successfully. Clients talk to CSAs directly more often. Very few clients have questioned the change. Three positions were eliminated. In addition, the team took on additional work without adding headcount. (SMART roll-out). Cycle time savings were estimated at 65% initially. The savings observed are about 50%. Critical factors made the change possible: Pilot to test client feedback Excellent communication to the team and clients about changes Extreme organization and focus

Focus areas – Completed Events MMS Screening & Validation MAS RFP Response mini kaizen Billing Initiation Process MAS Custom Project Process Performance Analytics Deployment Standardization Opportunity Management / CRM Improvement SAS Incentive Compensation Implementation Process Reduce time spent screening & validating data Improve quality & consistency of RFP responses Reduce admin burden for GTM resources Improve process workflows for custom deliverables Reduce time to implement Reduce admin burden Improve the process for IC implement-ations Objectives MAR 6-8 APR 16 MAY 1-3 MAY 15-17 MAY 15-17 JUN 11-12 JULY 17 -19 31% reduction in cycle time Process improve-ments imple-mented Identified 3 improve-ment work streams Results to be rolled out at July 10 report out Delivery time reduced by 25% Eliminated time spent entering detail into early stage opps Results

Process Priorities – Sample from Operations On an annual basis, lean leads and functional area management identify focus areas within each functional area.

Evidence of Cultural Entrenchment Leadership Lean continues to be a high priority for Seyed and team Lean Events U.S. has held more Lean events than any other country (110 events since Feb 2011) 30 additional events are in the pipeline Skill Development More than half of all certified Lean facilitators are in the US 19 certified and 48 in progress Results Over 25% of Lean’s global savings come from the U.S. Implementation is strong (>80% kaizen action items complete) Overview of the program and key projects including timeline and status Tracking metrics e.g. savings, efficiency improvements Our plan to go back and audit after the event was implemented (we may want to show at next ELG)

Lean Measurement & Reporting Tools Lean Event Executive Summary Template: http://arc.imshealth.com/gm/document-1.9.1639940 Guidelines for 30 / 60 / 90 Day Reviews http://arc.imshealth.com/gm/document-1.9.2276470 Lean PMO http://arc.imshealth.com/gm/leansigmaroadmap PMO Users Guide: http://arc.imshealth.com/gm/folder-1.11.1540558 Project Statistics (PMO Web Query): http://arc.imshealth.com/gm/folder-1.11.1517854 Guidance on standard metrics and metrics-monitoring tools Under development. Contact Tamra Kirkpatrick.

US & Canada: ISO Compliance In the US and Canada, the Kaizen management process is maintained in compliance with ISO standards Facilitators are required to comply with the Kaizen Management procedure http://arc.imshealth.com/gm/document-1.9.2241809 The procedure is written to provide a great deal of flexibility in selecting, planning and conducting kaizen events Process effectiveness is assessed: Results are reviewed in quarterly internal management reviews, and periodically, by external ISO auditors This slide is N/A to countries outside of the US & Canada. Reference to lean kaizen events are made in process documentation for other US and Canada processes; hence the need to bring our kaizen management procedure under ISO as well. A key benefit of ISO compliance: It requires on-going training of lean facilitators (training records must be maintained). Number of kaizen events per quarter, based on a rolling four quarters ≥ 15