1 CHAPTER 4 Data Warehousing, Access, Analysis, Mining, and Visualization

2 Data Warehousing, Access, Analysis, and Visualization What to do with all the data that organizations collect, store, and use? (Information overload!) Solution n Data warehousing n Data access n Data mining n Online analytical processing (OLAP) n Data visualization n Data sources Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

3 The Nature and Sources of Data n Data: Raw n Information: Data organized to convey meaning n Knowledge: Data items organized and processed to convey understanding, experience, accumulated learning, and expertise Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

4 DSS Data Items n Documents n Pictures n Maps n Sound n Animation n Video n Can be hard or soft Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

5 Data Sources n Internal n External n Personal Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

6 Data Collection, Problems, and Quality n Problems 1. Data are not correct. 2. Data are not timely. 3. Data are not measured or indexed correctly. 4. Needed data do not exist. n Quality: determines usefulness of data 1.Intrinsic data quality 2.Accessibility data quality 3.Representation data quality Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

7 Data Quality Issues in Data Warehousing n Uniformity n Version n Completeness check n Conformity check n Genealogy check (drill down) Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

8 The Internet and Commercial Database Services For external data n The Internet: major supplier of external data n Commercial Data Banks: sell access to specialized databases Can add external data to the MSS in a timely manner and at a reasonable cost Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

9 The Internet and Commercial Databases Servers Use Web Browsers to n Access vital information by employees and customers n Implement executive information systems n Implement group support systems (GSS) n Database management systems provide data in HTML, on Web servers directly Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

10 Database Management Systems in DSS n DBMS: Software program for entering (or adding) information into a database; updating, deleting, manipulating, storing, and retrieving information n A DBMS + modeling language to develop DSS n DBMS to handle LARGE amounts of information Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

11 Database Organization and Structure n Relational databases n Hierarchical databases n Network databases n Object-oriented databases n Multimedia-based databases n Document-based databases n Intelligent databases Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

12 Data Warehousing n Physical separation of operational and decision support environments n Purpose: to establish a data repository making operational data accessible n Transforms operational data to relational form n Data are transformed and integrated into a consistent structure Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

13 Data Warehousing Characteristics 1. Subject-Oriented. 2. Integrated. 3. Time variant. 4. Nonvolatile. 5. Summarized. 6. Not normalized. 7. Sources. 8. Metadata Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

14 Data Warehouse Architecture and Process Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ n May have one or more tiers –Determined by warehouse, data acquisition (back end), and client (front end) One tier, where all run on same platform, is rare Two tier usually combines DSS engine (client) with warehouse –More economical Three tier separates these functional parts

© 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang 5-15 Data Warehouse Development n Data warehouse implementation techniques –Top down –Bottom up –Hybrid –Federated n Projects may be data centric or application centric n Implementation factors –Organizational issues –Project issues –Technical issues n Scalable n Flexible

© 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang 5-16 Data Marts n Dependent –Created from warehouse –Replicated Functional subset of warehouse n Independent –Scaled down, less expensive version of data warehouse –Designed for a department or SBU –Organization may have multiple data marts Difficult to integrate

© 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang 5-17 Business Intelligence and Analytics n Business intelligence –Acquisition of data and information for use in decision-making activities n Business analytics –Models and solution methods n Data mining –Applying models and methods to data to identify patterns and trends

18 OLAP: Data Access and Mining, Querying, and Analysis Online analytical processing (OLAP) –DSS and EIS computing done by end-users in online systems –Versus online transaction processing (OLTP) Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

19 OLAP Activities n Generating queries n Requesting ad hoc reports n Conducting statistical and other analyses n Developing multimedia applications Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

20 OLAP uses the data warehouse and a set of tools, usually with multidimensional capabilities n Query tools n Spreadsheets n Data mining tools n Data visualization tools Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

21 Using SQL for Querying n SQL (Structured Query Language) Data language English-like, nonprocedural, very user friendly language Free format Example: SELECTName, Salary FROMEmployees WHERESalary >2000 Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

22 Data Mining for n Knowledge discovery in databases n Knowledge extraction n Data archeology n Data exploration n Data pattern processing n Data dredging n Information harvesting Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

23 Major Data Mining Characteristics and Objectives n Data are often buried deep n Client/server architecture n Sophisticated new tools--including advanced visualization tools--help to remove the information “ore” n End-user miner empowered by data drills and other power query tools with little or no programming skills n Often involves finding unexpected results n Tools are easily combined with spreadsheets, etc. n Parallel processing for data mining Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

24 Data Mining Application Areas n Marketing n Banking n Retailing and sales n Manufacturing and production n Brokerage and securities trading n Insurance n Computer hardware and software n Government and defense n Airlines n Health care n Broadcasting n Law enforcement Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

25 Intelligent Data Mining n Use intelligent search to discover information within data warehouses that queries and reports cannot effectively reveal n Find patterns in the data and infer rules from them n Use patterns and rules to guide decision making and forecasting n Five common types of information that can be yielded by data mining: 1) association, 2) sequences, 3) classifications, 4) clusters, and 5) forecasting Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

26 Main Tools Used in Intelligent Data Mining n Case-based Reasoning n Neural Computing n Intelligent Agents n Other Tools –Decision trees –Rule induction –Data visualization Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

27 Data Visualization and Multidimensionality Data Visualization Technologies n Digital images n Geographic information systems n Graphical user interfaces n Multidimensions n Tables and graphs n Virtual reality n Presentations n Animation Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

28 Multidimensionality n 3-D + Spreadsheets (OLAP has this) n Data can be organized the way managers like to see them, rather than the way that the system analysts do n Different presentations of the same data can be arranged easily and quickly n Dimensions: products, salespeople, market segments, business units, geographical locations, distribution channels, country, or industry n Measures: money, sales volume, head count, inventory profit, actual versus forecast n Time: daily, weekly, monthly, quarterly, or yearly Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

29 Multidimensionality Limitations n Extra storage requirements n Higher cost n Extra system resource and time consumption n More complex interfaces and maintenance Multidimensionality is especially popular in executive information and support systems Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

30 Geographic Information Systems (GIS) n A computer-based system for capturing, storing, checking, integrating, manipulating, and displaying data using digitized maps n Spatially-oriented databases n Useful in marketing, sales, voting estimation, planned product distribution n Available via the Web n Can use with GPS Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

31 Virtual Reality n An environment and/or technology that provides artificially generated sensory cues sufficient to engender in the user some willing suspension of disbelief n Can share data and interact n Can analyze data by creating a landscape n Useful in marketing, prototyping aircraft designs n VR over the Internet through VRML Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

32 Business Intelligence on the Web n Can capture and analyze data from Web n Tools deployed on Web Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

33 Summary n Data for decision making come from internal and external sources n The database management system is one of the major components of most management support systems n Familiarity with the latest developments is critical n Data contain a gold mine of information if they can dig it out n Organizations are warehousing and mining data n Multidimensional analysis tools and new enterprise- wide system architectures are useful n OLAP tools are also useful Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ

34 Summary (cont’d.) n New data formats for multimedia DBMS n Internet and intranets via Web browser interfaces for DBMS access n Built-in artificial intelligence methods in DBMS Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson, 6th edition Copyright 2001, Prentice Hall, Upper Saddle River, NJ