1. 14.5.20061 Statistics Seminar Forecasting Loads for a Telephone Call Center Sivan Aldor - Noiman Advisors: Prof. Paul Feigin Prof. Avishai Mandelbaum

2. 14.5.20062 Outline Call Centers Overview The Cellular Phone Company and Data Description The Mixed Model Evaluation Methodology Goodness of Fit tests Predicting Average Service Times Load Forecasts and Staffing Formulae

3. 14.5.20063 Call Centers - Overview Call Centers are the primary contact point between service providers and their customers in modern businesses. Call Centers comprise human agents who provide services, as well as an automated interactive facility which often handles the initial phase of the customer interaction, and which may provide complete service to a significant proportion of the incoming and outgoing calls.

4. 14.5.20064 Call Centers - Overview There are approximately 2.68 million operator positions in over 50,000 call centers in the US with some locations employing over 1000 agents. A typical call center spends between 60% and 70% of its annual budget on staff salary Call center investment in workforce optimization technologies will exceed $1 billion by 2006.

5. 14.5.20065 Project Data MOCCA Goal of Data MOCCA (MOdels for Call Center Analysis) Designing and Implementing a (universal) database and interface for storing, retrieving, analyzing and displaying call- by-call information from Call Centers Enable studying the behavior of Enable studying the behavior of: Customers Service Providers / Agents Managers / System Wait Time, Abandonment, Retrials Queues Lengths, Loads, Trends Service Time, Activity Profile

6. 14.5.20066 Project Data MOCCA The Wharton School Team, University of Pennsylvania Israeli Bank Israeli Cellular Phone Company Large US Bank Technion Team including the Statistical Laboratory group

7. 14.5.20067 The Cellular Phone Company The Company’s Call Center Working hours are 7AM-11PM Sunday-Friday 750 agents are employed on a regular weekday Handles 50,000-60,000 calls on regular weekdays Provides services to 21 different types of queues. The three largest are:   Private queue (30% of incoming calls)   Business queue (15% of incoming calls)   Technical Support queue ( 10% of incoming calls)

8. 14.5.20068 The Cellular Company Current Forecasting Procedure: An automatic “black box” system predicts weekly forecasts. Each Thursday the following procedure is carried out by the planning group: 1. Use 6 weeks of past data as the learning data 2. Predict the week which begins ten days later 3. Use subjective experience to correct the predictions 4. Derive the demand and required staffing for that week

9. 14.5.20069 The Cellular Company 16151413121110 9876543 2101234 56789 11 12131415161718 19202122232425 26272829303132 33343536373839 404142 SaturdayFridayThursdayWednesdayTuesdayMondaySunday Forecast Period Prediction Lead Time Learning Period

10. 14.5.200610 Data Description Data Description Arrival counts to the Private queue Each day divided into 33 half-hour intervals between 7AM-11:30PM The learning data includes working weekdays ranging between mid-February, 2004 and December 31, 2004 Irregular days and Holidays were removed All together, there are 254 days

11. 14.5.200611 Data Description Data Description

12. 14.5.200612 Data Description Data Description The call center has four different billing cycles Each of its customers belongs to one of the cycles. Each billing cycle is characterized by two periods: 1. 1.Delivery Period (1-2 days) 2. 2.Billing Period (usually one day), about a week later

13. 14.5.200613 Data Description Data Description SaturdayFridayThursdayWednesdayTuesdayMondaySunday 76543-D2-D2-D1-B1-B 14-B13121110-B9-D8-D8-D 21-B2019181716-D15-D 2827262524-D23-D22 31-D30-D29 Cycle 4 Cycle 1 Cycle 2 Cycle 3 B=Billing Period D=Delivery Period We model each cycle using two indicators one for each period August 2004

14. 14.5.200614 Billing cycles Selection Billing cycles Selection Do we really need all 8 indicators? We employ the following Poisson regression on the daily arrivals to determine the important indicators: Weekday effect Billing effect Delivery effect Daily Arrivals We tried different options…

15. 14.5.200615 Characterizing Billing Cycles Characterizing Billing Cycles Using the LR statistic, comparing to the full model, we arrived at the following two options: 1.Four 1.Weekdays, Four delivery period indicators and Cycle 1 billing period indicator 2. 2.Weekdays, One global delivery period indicator and Cycle 1 billing period indicator P-valueLR Chi-Square statistics 0.11215.99Option 1 0.07811.33Option 2

16. 14.5.200616 The Mixed Model We first use the following variance-stabilizing transformation to “change” the Poisson data into approximately normal data : For large enough since the transformed data follows Model assumptions: 1. 1.The arrival process follows an inhomogeneous Poisson process. 2. 2.During half-hour intervals the rate remains constant

17. 14.5.200617 The Mixed Model Fixed Effects Random Effects - No. of days between day i and day j

18. 14.5.200618 The Benchmark Model

19. 14.5.200619 Evaluation Methodology To test the prediction models use the out-of-sample performance measures. Predict each week (6 days) between April 12 and December 24, 2004. There are 203 days. For each week use the same data and lead time as the cellular company. At the end of the process we have for each period k during each day d:

20. 14.5.200620 Evaluation Methodology Evaluate the model using the following measures: 1. 1.For each period k during day d compute: 2. For each day summaries:

21. 14.5.200621 The Mixed Model – Billing cycles Benchmark1 Global Delivery+ Bill 14 Deliveries+Bill 1 33.2333.3334.26Lower Quartile 41.3940.4640.62Median 46.0544.5745.51Mean 54.3651.7253.21Upper Quartile RMSE Benchmark1 Global Delivery+ Bill 14 Deliveries+Bill 1 8.38.158.35Lower Quartile 10.469.9810.13Median 11.2411.0611.25Mean 13.4913.3713.81Upper Quartile APE

22. 14.5.200622 The Mixed Model – Billing Cycles Benchmark1 Global delivery+ Bill 14 Deliveries+Bill 1 0.390.880.85Lower Quartile 0.520.94 Median 0.510.92 Mean 0.6111Upper Quartile Coverage Probability Benchmark1 Global delivery+ Bill 14 Deliveries+Bill 1 51.57138.15134.87Lower Quartile 59.56157.22156.87Median 63.2161.71161.61Mean 70.07184.48184.79Upper Quartile Width

23. 14.5.200623 Weekdays patterns: 1. 1.Fridays are different 2. 2.Sundays are different 3. 3.All other weekdays are not significantly different during 76% of the periods. The Mixed Model – Weekdays Is it important to have 6 different patterns?

24. 14.5.200624 The Mixed Model – Weekdays Multi-PatternThree-Pattern 33.3332.65Lower Quartile 40.4638.25Median 44.5743.3Mean 51.7250.94Upper Quartile RMSE Multi-PatternThree-Pattern 8.157.83Lower Quartile 9.959.68Median 11.0610.8Mean 13.3712.86Upper Quartile APE

25. 14.5.200625 The Mixed Model – Weekdays Multi-PatternThree-Pattern 0.88 Lower Quartile 0.940.97Median 0.920.94Mean 11Upper Quartile Coverage Probability Multi-PatternThree-Pattern 138.15141.53Lower Quartile 157.22160.76Median 161.71164.33Mean 184.48186.79Upper Quartile Width

26. 14.5.200626 The Mixed Model – Within Periods Comparing different within-periods covariance structures 1. AR(1) 2. ARMA(1,1) 3. Toeplitz

27. 14.5.200627 The Mixed Model – Within Periods 50.7450.94Upper Quartile 43.2343.40Mean 38.4738.35Median 32.3932.65Lower Quartile ARMA(1,1)AR(1)RMSE 12.8312.86Upper Quartile 10.7310.8Mean 9.549.68Median 7.83 Lower Quartile ARMA(1,1)AR(1)APE 161.7185.4Upper Quartile 161.48162.84Mean 158.76160.76Median 138.98141.53Lower Quartile ARMA(1,1)AR(1)Width 11Upper Quartile 0.93 Mean 0.97 Median 0.88 Lower Quartile ARMA(1,1)AR(1) Coverage Probability Is ARMA really worth it? AR(1) Procedure takes 1 hourAR(1) Procedure takes 1 hour ARMA(1,1) takes 6-7 hoursARMA(1,1) takes 6-7 hours

28. 14.5.200628 The Mixed Model – Daily Effect How important is the random daily effect if we are predicting 10-days-ahead? We might expect it to have a very small influence on the predictions… We test two models: 1. 1.The Three-Pattern Model with the random daily effect 2. 2.The Three-Pattern Model without the random daily effect V d

29. 14.5.200629 The Mixed Model – Daily Effect 54.1250.94Upper Quartile 44.5343.40Mean 38.6338.35Median 32.0332.65Lower Quartile No DailyDailyRMSE 13.212.86Upper Quartile 11.0310.8Mean 10.099.68Median 7.867.83Lower Quartile No DailyDailyAPE 174.95185.4Upper Quartile 154.63162.84Mean 151.61160.76Median 136.12141.53Lower Quartile No DailyDailyWidth 11Upper Quartile 0.910.93Mean 0.940.97Median 0.850.88Lower Quartile No DailyDaily Coverage Probability When we carried out one-day- ahead predictions there were bigger differences

30. 14.5.200630 Testing for Goodness of Fit The Mixed Model Normal Assumption First we check the QQ-plot of the prediction residuals to see if it is normally distributed.

31. 14.5.200631 Testing for Goodness of Fit The Mixed Model Is the estimated value of this variance close to 0.25? We can use ARMA(1,1) to our advantage and check this out

32. 14.5.200632 Testing for Goodness of Fit The Mixed Model

33. 14.5.200633 Operational Regimes Efficiency Driven Quality Driven QED (Quality-Efficiency Driven)

34. 14.5.200634 QED Regime To predict the load (R) 1.We need to predict the arrival rate 2.We need to predict the average service

35. 14.5.200635 Average Service Time Patterns We fitted a quadratic regression based on the weekday and period The final model has the following components: PeriodPeriod*PeriodWeekday Weekday*Period Date

36. 14.5.200636 The Predicted Average Service patterns for a typical week: Average Service Time Patterns

37. 14.5.200637 Predicting Staffing Levels Given the user supplied we estimate as follows: predicted no. of required agents actual no. of required agents actual no. of required agents

38. 14.5.200638 Predicting Staffing Levels Averaging the difference separately for each period over the days we get the following graph:

39. 14.5.200639 Predicting Staffing Levels

40. 14.5.200640 1.Comparison with other models 2.Comparison with industry models 3.Trying to answer: “What is the effect of different time resolutions on prediction performance?” 4.Tying up some loose ends… Work in Progress

41. 14.5.200641 NOT

42. 14.5.200642 The Prediction Lead time What is the influence of the lead time on the APE and RMSE? Lead time (Days)