C.U.SHAH COLLEGE OF ENG. & TECH. WADHAWAN CITY DATA WAREHOUSE Guided by :: Mr. Tushar Jani Developed by :: Mr. Dipesh N. Doshi Mr.Pritesh B. Patel (4th sem) M.C.A.

A data warehouse is a repository of subjectively selected and adapted operational data which can successfully answer any ad-hoc ,complex ,statistical or analytical queries. it is situated at center of DSS of an organization and contains historical data both summarized and detailed information. Data warehousing technology is becoming essential for effective business intelligence ,business strategy , formulation and implementation in a globally competitive environment where larger amount of data processed faster. Data warehousing enables easy organization and maintenance of large data in addition to fast retrieval and analysis from time to time.

DATA MARTS From data warehouse , data flows to various departments for their DSS usage. The individual departments are called data marts. It is a subset of a data warehouse and is much more popular than data warehouse. Data warehouse is a collection of data marts.

Data Warehousing - Providing Data Access to the Enterprise organizations to provide their organizations flexible, effective and efficient means of getting at the sets of data that have come to represent one of the organization's most critical and valuable assets. Data Warehousing is a field that has grown out of the integration of a number of different technologies and experiences . These experiences have allowed the IT industry to identify the key problems that have to be solved.

Informational systems. The most important concept that has come out of the Data Warehouse movement is the recognition that there are two fundamentally different types of information systems in all organizations: Operational systems. Informational systems.

Informational systems: Operational systems: They are the systems that help us run the enterprise operation day-to-day. These are the backbone systems of any enterprise E.G :: Order entry, inventory, manufacturing, payroll and accounting systems. Informational systems: They have to do with analyzing data and making decisions, often major decisions ,about how the enterprise will operate, now and in the future.

Data Warehouse Architecture DWA is a way of representing the overall structure of data, communication, processing and presentation that exists for end-user computing within the enterprise.

The architecture is made up of a number of interconnected parts: Operational Database / External Database Layer Information Access Layer Data Access Layer Data Directory (Metadata) Layer Process Management Layer Application Messaging Layer Data Warehouse Layer Data Staging Layer

Data Warehouse Options: different dimensions that need to be considered for developing data warehouse: Scope of the data warehouse Data redundancy Type of End-users. Virtual" or "Point-to-Point" Data Warehouses Central Data Warehouses Distributed Data Warehouses

Developing data warehouse: Data warehousing technology has ability to organize, maintain large data and also be able to analyze in few seconds in manner and depth as required. Architectural strategies and organizational issues. Design. Data content. Meta data. Distribution of data. Tool for data warehousing. Performance consideration. Other requirements.

First build data marts before real data warehouse is built. Choose subject matter. Decide what the fact table represents. Identify and conform the dimensions. Choose the facts. Store pre-calculation in the fact table. Define the dimensions and the table. Decide the duration of the database and periodicity of updation. Track slowly the changing dimensions. Decide the query priorities and query modes.

DATA MINING The non trivial extraction of implicit previously unknown and potentially useful knowledge from data. KDD(knowledge discovery in databases) multidisciplinary field of research.

DATA MINING Vs QUERY TOOLS Data mining tools and query tools are complementary . Data mining does not replace query tools but it does give user a lot of additional possibilities.

PRACTICAL APPLICATION OF DATA MINING Lack of long term vision. Not all files up to date. Struggle between departments. Poor co-operations from electric data processing departments. Legal and privacy restrictions. Files are hard to connect for technical reason. Timing problem. Interpretation problem.

TECHNIQUES. Query tools. Statistical techniques. Visualization . OLAP. Case based learning(k-nearest neighbor). Decision trees. Association rule. Neural network. Genetic algorithm.