MDIT Data Warehouse Strategy Update Michigan Data Warehouse 2007 Annual User Group Conference November 6, 2007 Kenneth D. Theis Chief Deputy Director

On tap today More progress Going to the next level Governance The Score Card

More progress

180,000+ queries per day More progress Michigan currently is sharing over 2 terabytes of information, which equates to about 1/10 of all the books in the largest library in the world

- Michigan’s Statewide Data Warehouse - Data Sharing More progress More than 9,000 users in 5 major departments, 20 agencies and 100+ bureaus make use of the BI/data warehouse - Michigan’s Statewide Data Warehouse -

Going to the next level

Michigan Strategic Alignment Going to the next level The Governor’s six priority areas is the basis for… MDIT Strategic Plan with five goals Goal 2: Transform Michigan's services through sharing and collaboration Seven Technology Solutions Data Integration: is optimized through data warehousing

Business Intelligence going to the next level Prudent use of taxpayer money requires that we minimize the number of systems and tools that we use in Business Intelligence (BI) efforts The State will likely never have enough resources, but BI can help to use existing resources more efficiently and effectively The goal of BI is to maximize the use of shared or common resources by reducing the number of BI platforms to one, and to provide a single tool for each class of BI applications Data Warehousing is the physical aspect of storing data, and the process of turning data into information and knowledge is Business Intelligence

Gartner Study Going to the next level Gartner was contracted by MDIT to help the State of Michigan develop an enterprise information management and data warehouse strategy to: Increase the ability to better support both the infrastructure and the delivery of Business Intelligence solutions to the enterprise. Increase the ability to leverage both infrastructure and expertise. Actionable Increase information sharing across government.

Governance

The Next Steps Governance

Business Intelligence Competency Center Governance

BICC Mission Governance Encourage and enable business intelligence in the State of Michigan It will do so by: Supporting data sharing, Providing support and training for data analysis, Promulgate and support standards

BICC First Steps Governance OES: Privacy/Security BICC Data sharing Metadata support DW architecture Development support Communication Training BI development Agency Oversight OE&FS: Budgeting Agency Services Development An oversight group will be established to enhance communication with the agencies and give them a voice in business intelligence standards and directions. The data sharing function will provide basic support for data sharing by: . Inventorying data sharing agreements across the state . Making these agreements visible within state government . Creating a template for agreements . Working with agencies to assist them in creating agreements . Manage standards associated with data sharing The metadata support function will create a repository of information about the data we use in the state, including what it is, where it comes from, who owns it, and how it is used. It will begin with the data already being shared and on the warehouse. As the function progresses it will be a service available to any agency that needs to define, or understand the data that it is using. As the state moves toward Service Oriented Architectures this repository will be a major asset. Data Architect/Development Support will work with the agencies to create the template for our “Master Data” Warehouse. Jim Tepin’s talk on the “Core Common Models” provides an example of how this could work. In addition the BICC will provide support for agencies that are new to data warehousing by working with them to move their data onto the warehouse and build systems to keep it updated. This function will be responsible for data/platform standards. Communication/Training. In order to be successful, people in DIT and the other agencies need to know what the BICC is doing. This function will ensure that BICC Services and standards are publicized so that agencies can take advantage of them. In addition this function will coordinate training for the skills and knowledge required to take advantage of the data warehouse. Very few agencies have the skills needed to take full advantage of the data on the warehouse. While the focus of the BICC will be to enable agencies to do their own analyses, some data requests will take more skill to accomplish. That will be the function of the BI Development position. Agency data stewards will be responsible for understanding their data and working with the BICC to ensure that it is defined and used correctly. Privacy and security of the data on the warehouse are paramount. As such the BICC will work closely with the Office of Enterprise Security to ensure that all actions are taken to protect them. Budgeting is required for all groups and that will be coordinated with the Office of Employee and Financial Services The DCO will continue to administer the data warehouse proper, with policy and development support moved to the BICC Where Agency Services has the capacity they will continue to do the majority of the Data warehouse maintenance and BI development. The BICC will support and supplement their efforts. Where the Agency Services organization does not have the capacity the BICC will work with them to built it and work side by side to develop their warehouse and BI Support. When needed the BICC will create new BI applications or high level queries. Agency Data Stewards DCO Warehouse DBAs

The Next Steps Governance Data Sharing inventory and build data sharing templates Enhance the process to share existing data on the data warehouse Platform evaluation – next generation Begin design of the “Master Data Warehouse” In order to better manage data sharing we have to understand what is being shared and how it is being shared. As the data is collected, it will be made available so that agencies know exactly what data is being shared and who is responsible for that data. In the past parts of an agency have initiated new agreements for data that other parts of the agency have already acquired and to which they have legitimate access. By providing a statewide repository of these agreements, the BICC will enable agencies to understand what is already being shared and who they need to contact for access to the data. Much of the state’s data sharing efforts to date have been informal. One of the primary reasons for this is that it is too hard to set up agreements. Staff do not know how to do it and what needs to be included. The BICC will assist with this by using information from the existing agreements to create templates for use in future. Beyond this the BICC will work with the agencies to facilitate data sharing efforts. Data already on the data warehouse has solid technical support for sharing. Beyond just sitting on a common platform, the VCentral concept in use on the data warehouse allows agencies to share their data without giving up control. However, just because the data is on the warehouse, doesn’t mean that it can be shared. It still requires the approval of the agency that owns the data for another to use it. That’s straight forward. But much of the data on the warehouse is maintained by organizations that got it from another organization. They don’t own it, but they need to be involved in any sharing decision since they control access. This is not their job and they rightly see it as a distraction from what they need to do. The BICC will work with these agencies to find ways to get them out of the loop, and encourage authorized reuse of the data. We’ve used the Teradata data warehouse since the early 1990’s. It has been a good platform and performed well for us. But we have not looked at anything else in that time period. That’s caused some tension between agencies who prefer other platforms. In addition the market has changed in that time period. We need to look around and find the best platform for all of the state’s data warehousing and business intelligence needs. So we will launch an RFP in the next year that will ask the primary data warehousing vendors to come in and give us their best shot at a new platform—one that will provide at least as well as what we have and preferably better for the least amount of money. All agencies who do data warehousing work will participate and we will use this to choose a new platform, which may well be Teradata, or IBM or Oracle, to carry us forward for the next 10 years. The whole goal of the BICC is to connect the silos and encourage information sharing. That is what the Master Data Warehouse is about. We have legitimate lines of business in the state of Michigan and it will be years, if ever, before we will be able to provide a single consistent view of all of their data. But there are consistencies between the silos. Most of our agencies deal with citizens and businesses. All of them have addresses. The Master Data Warehouse will pull these common areas together to provide a consistent source for that common data. This is going to be an evolving process, we will start with pure addresses, add a citizen view, then a business one. As we move forward we will find further common areas, and as we understand them and work with the agencies we will add them to the core system. This will keep us busy enough for now. In the future, we hope to expand the services of the organization to further encourage agencies to take full advantage of business intelligence.

BICC Data Sharing Opportunities Governance

The Score Card

The Score Card 2005-2007 2008 Develop Strategic Plan Establish BICC within Shared Services Continue adding and integrate data to the Data Warehouse Complete the foundation for the “Master Data” Warehouse Consolidate data marts Establish the BI platform for the next 5-10 years Create Center of Excellence for data warehousing Create data sharing inventory/establish data sharing support Empower business users through BI tools and training Improve management of shared data on warehouse Improve the Data Warehouse cost structure

The Score Card 2009…Game On!

Kenneth D. Theis, Chief Deputy Director Michigan Department of Information Technology