What is Data Warehousing All About? /7/2012

Agenda  Intended Audience  Introduction to the 2012 update  What do you want, really?  What is the Data Warehouse all about?  Management Decision Making Process  Role of the Data Warehouse  What are people saying about Data Warehousing?  What went so wrong?  How to make it go right?  Business benefits of Data Warehousing  What works?

Intended Audience  As much as I would like business managers to listen to this I realise that the actual audience will be BI IT Managers and professionals  That is kind of sad because it is business managers who need to hear this presentation.  Most simply will not take the time to listen  They are “too busy” to listen and then they wonder why they do not have successful BI projects  It takes VERY little time for a business manager to listen to what he/she needs to know to make BI successful  When I give an abbreviated form of this presentation to business managers they “get it”  IT Managers?  I will make a second version for your business managers.  It is up to you to get them to listen to it.

Introduction to the 2012 Update  The original version  This presentation was written in 1994 based on a presentation by Keith Fountain of Metaphor in 93.  Keith made this presentation to many prospects in Sydney in when I invited him to come and talk to our IBM Insurance Clients in Sydney  I upgraded versions of this presentation for 10 years  Now, in 2012, I have upgraded it because it is as relevant today as it was nearly 20 years ago  This is a valuable presentation that belongs in the public domain in the you tube age  We have not moved forward nearly as far as I expected in Data volumes yes. Data models yes. But the ability/desire of business people to exploit the data warehouse has gone down, in my opinion

What Do You Want, Really?  Improved profitability with sustainable growth  Attention to both revenue and costs  Improved decision making capability  Make more effective decisions more often  Consistent measures of business performance  Profit is a fact, not an opinion  Compliance to government legislation  You can be put out of business if not  Others?

What is Data Warehousing All About?

What is the DW All About?  The Data Warehouse is all about “Supporting the management decision making process”  It’s that simple, and that complicated

Management Decision Making Process How’s Business? Why? What if? Invest $$$

What is the Data Warehouse? (1993)  Time Variant  Subject Oriented  Integrated  Contains non volatile snapshots  A Data Warehouse is a collection of data in support of the management decision making process.*  This definition has stood the test of time remarkably well. It is as relevant today as it was in  This definition has stood the test of time remarkably well. It is as relevant today as it was in *Source: Building the Data Warehouse by W.H. Inmon

What Does a DW Look Like? (1993) Highly Summarised Lightly Summarised Current Detail Older Detail (Tape/cdrom) MetaDataMetaDataMetaDataMetaData

What Does a DW Look Like? 2012

Role of the Data Warehouse How’s Business? Why? What if? Invest $$$

What are People Saying About Data Warehousing? Please remember this was in the 90s.

What are People Saying?  “The business case for warehouses is simple: they help turn data into a competitive tool” COMPUTERWORLD

What are People Saying?  Tom Peters  We are living through a shift from selling virtually everyone the same thing a generation ago to fulfilling individual needs and tastes... by supplying... customised products and services. The shift [is] from “get the sale now at any cost” to building and managing... databases that track the lifetime value of your relationship with each customer.

What are People Saying?  Rapp & Collins  it may almost be time to replace “location, location, location” with “database, database, database”.

What are People Saying?  Stanley Davis - Future Perfect  Mass Customisation  Standardise the Commodity and Customise the Service that Surrounds It  Example: Telephone calls

What are People Saying?  Philip Kotler - Marketing Management  Mass markets are fragmenting into micro- markets; multiple channels of distribution are replacing single channels... The winners are those who carefully analyse needs, identify opportunities and create value- laden offers for target customer groups that competitors can’t match.

IDC  Perhaps the single most important ongoing occurrence to affect data warehousing has been the information explosion. Organisations realise that, given the fundamental relationship between knowledge and power, utilisation of this information is key to their competitive positioning.

Meta Group  Bottom Line: Data warehouses are an increasingly critical component of the systems that support the ever-increasing tempo of business competition.

Let Us Take a Checkpoint  We just saw some very strong comments made in the 90s about data warehousing.  I made MANY similar comments myself.  Who, listening to this, can honestly say their company has been able to gain very significant competitive advantage and sustain that competitive advantage via data warehousing and business intelligence?  In my experience this is very, very few.  With the consolidation of BI vendors it will be even fewer in the future.  WHY were the promised benefits not realised?

Let Us Take a Checkpoint  In my opinion not enough senior business managers listened closely enough to those of us who knew what we were talking about  In the late 90s every man and his dog was a “BI Consultant” whether he knew how to spell “BI” or not  I tried my best to put standards into the public to show people what a BI consultant should know  Men like me were widely ignored as we tried to bring in standards  BI has become “write more useless reports” in the main  Many people have become enamoured with “sexy charts”  Many people no longer look for competitive advantage and therefore they do not find it  We have snatched defeat from the jaws of victory in the vast majority of BI Projects  The 00s were a frustrating time for me watching the BI Industry be given a bad name by failure after failure by people who could not spell “BI” let alone build a data warehouse

Case Studies?  I am not going to bore you with case studies  Every vendor has “glamorous case studies” with promises of great things to come  Alas, mostly those promises are not delivered  Make no mistake SOMETIMES there is success  One of my clients paid for the project before it went live!  One of my clients doubled gross profit inside 2 years  One of my clients called it the most successful project ever  But success has been “surprisingly” elusive  Well…it has not been a surprise to me  I can usually tell if a BI project will deliver success by reading the requirements document and or project plan  It is THAT obvious if the dev team know what they are doing

What Went So Wrong?

 In March 1991 I saw the future  It was called the Data Interpretation System from Metaphor Computer Services (  It took me less than 30 minutes to realise this  I understood that my life had just changed  I was about to embark on another career change  I would move from getting data into computers to getting knowledge OUT of computers  I would move into a career where gleaning knowledge from masses of data to make decisions would be my job  And for 5 years it was  Then what happened in many cases was we built the data warehouse and the customers said “thanks for that, we will do our own data analysis now”. And they produced more reports.  SOME customers did very well. Many did not.  Those that did not are the ones who claim BI is not very good.

What Went So Wrong?  People refused to learn what the data warehouse was all about in the first place. Here it is again. From the late 80s. The most elegant definition of a data warehouse ever written down.  “A Data Warehouse is a collection of data in support of the management decision making process.”  Yet? I can go to 100 DWAs and ask the question “can you explain to me the Management Decision Making Process?” and they will look at me like I just arrived from the moon.  They will often say “What has data warehousing got to do with the MANAGEMENT decision making process?”  Almost NO ONE in BI can properly articulate what the “Management Decision Making Process” is  I find that quite amazing  Go ahead. Ask your “BI Consultants”. See how they go

How to Make it Go Right?

 Metaphor history explained  This is best explained in two anecdotes  Anecdote 1  A major Australian company with two failed BI projects  In the business requirements discussion a comment was made  “Do you realise the product is different every week?”  This comment was checked and found to be true  The ERP could not implement allocations on selected weeks  Allocations were implemented in the DW and fed to the ERP  Anecdote 2  Qantas Cargo had two failed BI projects  I was engaged to do “third time lucky”  Complaints were made about me…situation normal  The CEO of Cargo gave me the chance  We delivered an outstanding prototype and the project went on to be very successful

How to Make it Go Right?  What was different about what I did?  I was focussed on the “management decision making process”  I looked for the decisions being made in the company that affect both revenue and costs  I gathered the data that would support those decisions both plan and actual  This is NOT rocket science  But as far as I know, VERY few people do this  Ask you consultants for PROOF they do this  Consider replacing them if they fail this test

How to Make it Go Right? (90s)  Success was virtually assured if the DW was aimed at the “management decision making process”  Every time a management decision was being made, data from the data warehouse should be mandatory  This means PLANS and FORECASTS are a natural and REQUIRED part of the DW  Plans vs actuals are NOT optional in a good DW  Capturing reasons for variance is not optional  Multiple versions of plans is not optional  Recording investment decisions is NOT optional  Being able to run “business process change” tests is not optional  Being able to run test campaigns is not optional  Recording everything you can lay your hands on is VERY desirable

How to Make it Go Right? (10s)  All the above PLUS  Have a very good data model  The data model is the foundation  Have a very good idea of the applications to build  Preferably pre built and easy to customise  Have segmentation engines to put objects (usually customers) into segments  Disconnection of the components  Make customisation fast and easy  DO NOT re-invent the wheel  Have a methodology that is consistent with the tools  Focus on supporting decisions wherever smart decisions might generate business benefit  Put less emphasis on operational reporting

Questions to Ask Vendors  Get them to explain the “Management Decision Making Process” to you. See how they go  Ask them to explain their methodology and how it works and why it is the way it is  Ask them to explain their data models and applications  If they are talking about “we start with a blank sheet of paper” then you should be VERY wary as to why that might be  Ask them about business process change testing  Ask them about campaign management and testing  Ask them about segmentation and how that works  Ask them about customisation of existing apps  Ask them about how they liaise with management  Make them talk about HOW they achieve results and WHY those results are achieved  Ask them how it could be you would RETAIN your competitive advantage if you hire them

Questions to Ask Vendors  Ask them about what recognised business frameworks their BI solutions fit into  We use “Strategic Information Alignment Framework” from Donald Marchands book Competing with Information Add Value Customers and Markets Create New Reality Intelligence (Social, Political, Technological etc) Reduce Costs Transactions and Processes Minimise Risks Market, Legal, Financial Operational

Questions to Ask Vendors  If you are unsure if your BI vendors can deliver then feel free to hire IBI to advise and guide you on your BI project  For a small retainer we can review the vendors deliverables and tell you if you are getting value or not  Example.  A recent client spent USD2 M on a project that failed  As soon as I saw the proposed project plan I knew the vendor could not be successful  Yet the same vendor wanted a steep discount on my rates because “we do not have enough money to pay you”  I have done my best to give companies world wide good advice over many years  Mostly men like me are ignored and BI project failures continue to happen at high rates  Skill levels of the people doing the projects being the #1 cause

Business Benefits

A Way to Test New Ideas  How do you test new ideas for profitability?  Data Analysis/Mining “tests” ideas  Data Mining to find “diamonds”  There are “diamonds” in your data  Marketing Data Warehouse  Small investment, fast return  Likely source of “diamonds”

Business Benefits  More cost-effective decision making  Better business intelligence  Enhanced customer service  Enhanced asset/liability management  Aligned with corporate downsizing  Relationship to Business Process Reengineering

What Works

Likely High Payback Projects  High Volume, Low Cost products  Telco, retail, web, media (BI4ALL Models)  Marketing Campaigns  Customer Value Index  Customer profit contribution  Tiered servicing of customers  Product launch, modify, termination, pricing  Customer/Product understanding  Segmentation for marketing

What Works  Senior Manager(s)  Making big decisions on scant data  Demanding information  Marketing Manager(s)  Demanding better sales targeting  Do something you can’t do now What works is generally not so easy to do

What Doesn’t Work  IT technology driven project  Let’s try this new box, tool, toy...  Pie in the sky wishful thinking  There seems to be a lot of this about  The strange idea that dashboards will solve the problems of management  DW as an answer to operational reporting

Places to Start  75% of DWs start in:  Marketing/Sales  Customer Information Systems  All benefit from integrating data from multiple source systems  All contribute to new revenue and profit  Most of the other 25% start in  Performance measurement projects  Financial reporting and analysis

Places not to Start  Financial Systems  eg. Ledger, Accounts Receivable, Product Catalogue  Not customer focused  These systems are already ‘good’  Provides ‘better’ numbers for accountants  Little opportunity for new revenue

Cost Justifications  These are hard to do in advance  If Management is not convinced in the value of BI better to just leave it alone in 2012  It is hard to believe there is a business manager alive today who does not know there is value in BI  This is as opposed to being able to get that value extracted which is a different thing

Summary  Intended Audience  Introduction to the 2012 update  What do you want, really?  What is the Data Warehouse all about?  Management Decision Making Process  Role of the Data Warehouse  What are people saying about Data Warehousing?  What went so wrong?  How to make it go right?  Business benefits of Data Warehousing  What works?

Thank You for Your Time!