1. The Power of Analytics Applying and Implementing Analytics – How to, When to, and Why May 23, 2016 Session 2 Presented by Kelly Jin Citywide Analytics Manager City of Boston Analytics Implementation Case #3: Operationalizing Predictive Analytics at the City of Boston

2. The City of Boston is home to over 650,000 residents, 120 schools, and 850 miles of streets. The City is one of the largest employers in the metro Boston region, employing over 17,000 employees, with the majority made up of public safety officials and teachers. The City of Boston operates with a strong mayor form of mayor-council government and is made up of 14 Cabinets that comprise the Mayor's Cabinet, similar in structure to the Federal government agencies that make up the President's Cabinet. Each Cabinet is headed up by a Chief. Mayor Walsh formed Boston’s Citywide Analytics Team in January 2015 to bring the power of data and analytics to decision-making across City Departments. The Analytics Team is made up of 15 members, with roles including business analysts and data scientists. The Team is currently working on several predictive analytics projects, one of which is predicting overdoses for intervention and recovery services. The predictive analytics projects in their current form are multi-month projects led by an academic fellow. As this is the first project, the team has yet to operationalize the results, recommendations, and processes within City departments. What are our analytics project objectives? Analytics Implementation Case #3: Operationalizing Predictive Analytics at the City of Boston Several overarching questions exist for the project: Armed with an algorithm that may assign a risk factor to individuals who overdose, how do we most effectively operationalize the algorithm (hiring/internal, use of fellows)? What is a minimum viable product we could present back to Boston Public Health Commission (BPHC) and EMS on intervention? What before and after metrics should we track? How do we justify the investment, and what kind, to leadership? What skills do we need to manage this kind of work? (hiring, developmental) Case Introduction 2

3. Current Situation at City of Boston Analytics Implementation Case #3: Operationalizing Predictive Analytics at the City of Boston Predictive analytics projects are typically longer-term projects, requiring access to larger, more complex and sometimes multiple datasets. For analyzing overdoses, the City worked with the public health department and Emergency Medical Services (EMS) on signing data sharing privacy and security agreements due to HIPAA. The academic fellow involved completes all analysis on-site at the public health department to adhere to HIPAA guidelines as the data used for the analysis is EMS data that is case (individual) level and contains sensitive and private information. EMS has an existing business process and filter to identify Narcotics Related Incidents (NRI). Additionally, the data is a 3 year extract up to October 2015. The data includes such fields such as: name, gender, SSN, date of birth, and location. Currently significant work must be completed to link records of repeat patients in a systematic manner and geocode EMS pickup locations. Most EMS records do not have unique identifiers to enable tracking repeat patients over time. Data is manually entered, and typos or misspellings are very common, making simple matching difficult. Much of the summaries are manually tallied for reporting. As a significant amount of effort is required to clean the data extract, an opportunity exists to work with BPHC and EMS on the data intake side, and potentially in the future to integrate (anonymized) data into the City’s data warehouse to compare different data sets (ex. homeless client and housing data) 3

4. Challenges, Risks, and Roadblocks at City of Boston There is limited budget and resources for the Analytics team to hire staff with a more advanced data and analytical skillset, typically required to complete a predictive analytics project (R, Python, machine learning). Academic fellows stay for a 3 to 12 month project with the Analytics team. Fellows from academia typically do not have experience in communications, stakeholder, and change management, or past experience working in municipal government. At the end of the project, the fellow and team must deliver the results and recommendations to executive leadership and ultimately transition the knowledge and processes to full-time employees. Resources are lacking within departments to not only implement and operationalize the recommendations and processes (for example, once presented a list of individuals or places to intervene, who will implement changes to existing processes and train staff as needed?) but also to measure performance improvements over time. Average salaries for data scientists in city government are > $100K annual salary, with comparable salaries for data integration managers who manage ETL processes and infrastructure. Hiring timelines are typically 3-6 month processes. Analytics Implementation Case #3: Operationalizing Predictive Analytics at the City of Boston 4

5. Case Solution - Provide the following: 1.Recommendations: What are the 3-5 priority activities/actions the City should undertake to effectively operationalize insights from predictive analytics on potential drug overdoses? 2.What will these activities cost the City (ROM only)? 3.In what sequence or priority should these activities be planned and/or executed? 4.What other key considerations should the City take into account? Analytics Implementation Case #3: Operationalizing Predictive Analytics at the City of Boston 5