An Economic Cause of Data Strategy Failure Richard Howey Founder and Principal, Data Value Services LLC rich@datavalueservices.com Doctoral Candidate, Metropolitan State University dj4392zd@metrostate.edu

Data Sharing is an Important Goal of Data Strategy Many businesses need to share data across organizational boundaries inside the business enterprise Data Strategies are often intended to enable that sharing

The Negative Impact of Poor Data Sharing So many copies of the same data is expensive! Where is the data that I need? Why do we get different answers to the same question? Database Extract Program Source: Inmon, W. (1996). Building the data warehouse, second edition. New York, NY: Wiley.

Many Data Management Technologies Enable Data Sharing… XML DW/BI ODS DaaS MDM/RDM EAI JSON

… But the Success Rate is Low Source: http://Ambysoft.com/surveys/datawarehousing2013.html Source: http://www.sirsol.com/blog/2013/11/Why-Master-Data-Management-Efforts-Fail

Have We Uncovered the Real Root Cause? Most proposed solutions focus on the technology… …or the project What about the business environment in which IT operates, projects execute and data must be shared?

Data Sharing is a Form of Corporate Resource Allocation How does your business enterprise manage the allocation of resources across business units?

Two Methods of Managing Resources in a Business Top-Down Management Internal Market Mechanism

Internal Markets – Business Within a Business Market: A system that “allows individuals to exchange goods and services voluntarily, based on prices, without knowing one another” Source: “Market Theory” in Encyclopedia of Science, Technology and Ethics (vol. 3). 2005. Detroit, MI: Macmillan Characteristics: Competition Chargebacks for goods and services Management rewards based on internal market success

Question Does the “invisible hand” of a market always produce an optimal supply of a good or service?

Efficient Markets and Market Failures (Market May Function) Private Good or Service (Efficient Market) High Rivalry Public Good or Service (Market Failure) (Market May Function) Low Low High Excludability

Market Failures for Public Goods or Services Externality – Effect of a market transaction on a third party who does not participate in the transaction Non-Rivalry and Non-Excludability -> Externality Third parties get to benefit as “free riders” without paying Removes incentive for anybody to voluntarily pay Typically financed through taxes and supplied by government Classic Example: National Defense Would you be willing to voluntarily fund your country’s military budget from your own pocket if you weren’t taxed for it?

Private Good or Service Where is Data? Corporate Management will want data to be here to maximize value of shared data “Buyers” and “Sellers” of data will want it to be here to make market functional (Market May Function) Private Good or Service (Efficient Market) High Rivalry Public Good or Service (Market Failure) (Market May Function) Low Data Low High Excludability

Is There a Data Excludability Paradox? Corporate policies and culture that reward managers for internal market success tells managers with control of data resources to make the data excludable, i.e. withhold data Corporate sponsorship of data sharing initiatives, e.g. DW/BI, MDM, etc. tells managers with control of data resources to make the data non-excludable, i.e. share data Can you blame a manager whose pay and career depend on obeying the first message for deciding to obey it rather than the second message?

Symptoms of the Paradox Managers who need to use shared data: “We need to build a [enter organizational silo name here] [data warehouse, MDM system, etc.]” “I won’t pay to fix the enterprise data warehouse but I will pay for my own system” “That [MDM, DW, Metadata, etc.] system is a great idea – so long as I don’t have to pay for it” Managers who need to supply shared data: “We’ll help you get that data from our systems … in the 4th quarter 2 years from now” “It will cost you [enter exorbitant amount here] for us to help you get that data from our systems”

Why Is It Important to Know that this Behavior Comes from a Market Failure? “No decentralized pricing system can serve to determine optimally levels of collective consumption.” Paul Samuelson: First American to win the Nobel Prize in Economics (1970) Source: Samuelson, P. (1954). The pure theory of public expenditure. The Review of Economics and Statistics, 36(4), 387-389.

The Solution MUST Come From Centralized Corporate Leadership It is theoretically impossible for business unit management, project leaders, etc. to solve data withholding behavior problems without centralized intervention If you have repeatedly had poor results with data sharing initiatives and nothing your projects have tried solved the problem, then the data exclusivity paradox may be the root cause

Corporate-Level Solution Ideas Corporate policies must make data sharing more attractive to business unit managers than data withholding Fund data sharing initiatives from an enterprise budget Reward business unit managers who contribute to enterprise-level data sharing initiatives Do not hold business unit managers accountable for not meeting business unit goals if the reason was that they put resources into enterprise-level data sharing Give credit (bonuses, subsidies, etc.) to business units who share data that is used by other business units Create and enforce data sharing contracts between business units to make supplying data to other parts of the business part of the normal job Implement an executive-level data sharing/governance/strategy committee that makes the final decisions on data sharing conflicts in the best interest of the enterprise overall

Summary Internal markets are a good way to manage the allocation of many resources in large, complex 21st century businesses Unfortunately, data is not one of those resources Because data is non-rivalrous and preferably non-excludable within the borders of the business, an internal market will fail to optimize the supply of data This market failure may be a reason why so many data sharing initiatives repeatedly fail no matter what technology or project structure is used The market failure can only be corrected by centralized corporate management action

Upcoming Research A web-based survey to collect data on this phenomenon will be conducted during the winter of 2015-2016. These data will be used in a doctoral dissertation that will explore the degree to which this effect actually occurs in practice. If you would like to participate in this research, please email Richard Howey at: dj4392zd@metrostate.edu

Questions?