Lecture 3: Measurements and MOP (Measures Of Performance) Service Engineering Galit B. Yom-Tov

Why are measurements important? Improving service process – Understand and analyze the service process – Measure influence of changes Manage and control service Examples: KeyCrop, RFID in hospitals (2 movies), Transportation, Retails

KeyCrop: Service Excellence Management System (SEMS) [Kotha et al. 1996] Over 1300 branches, 210 million customer-teller transactions per year. The SEMS models measure branch activities and generate reports on customer wait times and teller proficiency and productivity levels. Results and impact ( ): For branches using the models. Dissatisfied customers: 17%->8% Customer processing time reduced by 53 percent (246 sec->115 sec) Percent of customers waiting more than five minutes: was 14% -> 4% % of branches with 90% of customers waiting less than five minutes: 42% - > 94% % of days with 90% of customers waiting less than five minutes: 55% -> 89% Cost $500,000 (including training). Expected Savings $98 million over five years (estimated 95-99).

Real-time Control RFID at hospital Traffic control Supermarket lines

MOP - Measures Of Performance Customer perspective Server/Manager perspective

MOP - Customer perspective How long the customer wait? Average time in queue, Probability of waiting. Was it too much? Probability of abandonment, LWBS, Reneging. Surveys. How long was the actual service? Average service time. Did the service actually finished to the customer’s satisfaction? Probability to return to additional service. “I was promised that an agent will come back to me in 1 hour.” Did they come back on time? Probability to return on time, Tardiness. Was the promised time too long? Reneging, Probability the customer called again before we got back to him.

MOP - Server/Manager perspective How many calls the agent answers during a day? Arrival rate, Average service time. Was is too much? Utilization, Agents turnover rate Do we use our representative efficiently? Utilization How many customers are waiting? Queue length Do we treat VIP customers as such? Waiting time by customer type

MOP and Organization Goals Connect MOP to Goals: Service Level Agreements: Call center Regulations: Hospitals (UK, Singapore), Call Centers (Israel) Measure are indicators of the operational status of the system, and correlated to higher level goals

Measurements Data is the Language of Nature Prerequisite for Science, Engineering and Management, yet Empirical “Axiom” = Problems with Historical Records – The data you need is not there for you to use: Not collected or erased, contaminated, … – If there is data, it has ‘frequencies” but no “times”: Fires, Courts, Hospitals, Projects, … Fires – If “times”, typically aggregated means but no std’s: Let alone histograms / distributions, Typically small samples, too short time-periods Often paper-archives, not computerized

Measurements Challenges – not Technological – Too little: “Complete” Data (KeyCrop) – Too much: Transaction mgt., Data Mining

“Production” Of Justice ≈ in months / years ≈ in mins / hours / days

“Production” Of Justice “Some judges are not efficient and have too high backlog of cases” Can we compare judges?

Judges: Operational Performance - Base Case

Judges: Performance by Case-Type

Judge: Performance Analysis

Obtaining Data - Face -to-Face Traditional work measurements – Stop-watch: utilization profile, times I-method Network – Ticket upon arrival (#,type) – Sensors at servers – Diagnostic/Research-device S-method 1-station – Tilets upon arrival (#,type) – Queues logically orders, sitting C-method 1-station – Sensors of arrival and service-starts – Queues physically orders, standing F-method 1-station/network – Transactions (automatically) recorded – Off-line (end-of-day) and Real-time – Inference of missing details – RFID Online global control exists (e.g. KeyCrop)

Obtaining Data - Telephone IVR/VRU ACD/CTICRM Additional data: HR Surveys Call recording

Data examples Changes over time Resolution Histograms Pareto Fishbone diagrams And why averages don’t tell the whole story

Call Center Data

Weekly arrival rate to a call center

Daily arrival rate to a call center

Arrival Process (1999)

Remarks on arrival rate Endogenous or exogenous? – Appointment: time interval/specific hour – Service policy: FCFS, Triage, Due Date (Flight) – Company actions: Billing – Service level: waits, returns

Service time by hour of day

Service Time Distribution – Call center

Service Time Distribution – call center

Service time distribution – Government office

Service time distribution – Internal Wards, Daily resolution

Service time distribution – Internal Wards, Hourly resolution

Arrival & Departure from IW

Service time distribution – Maternity Wards, Hourly resolution

Delays in Transfer: ED to IW

Delay in Transfer by Patient Type

Transfer process: ED to Internal Wards

Causes of Delay in Transfer Fishbone diagram Pareto

LOS by transfer hour

Pareto Chart

Statistical Survey – Rescue & Fire fighting commissionership (January-September 1992) Back