Data Mining - Introduction Compiled By: Umair Yaqub Lecturer Govt. Murray College Sialkot

Motivation  A result of natural evolution of information technology Mature database technology  1960s: Data collection, database creation, hierarchical and network database systems  1970s: Relational data model, relational DBMS implementation  1980s: RDBMS, advanced data models (extended-relational, OO, etc.) and application- oriented DBMS (spatial, scientific, engineering, etc.)  1990s: Data mining and data warehousing, multimedia databases, and Web databases  2000s: Stream data management and mining, Web technology (XML, data integration)

Motivation (contd…) Advances in digital hardware  Digitization Images, sounds, videos  Miniaturization of digital processors Ubiquitous devices capturing and communicating data  Digital storage Increase in capacity and reduction in cost

Data Mining & Data Analysis  Watch out: Is everything “data mining”? Handling small amounts of data Simple retrieval and deductive query processing Data Mining Database Technology Statistics Machine Learning Pattern Recognition Algorithm Other Disciplines Visualization

Data Mining: On What Kinds of Data?  Database-oriented data sets and applications Relational Databases Data Warehouses Transactional Databases Advanced datasets and applications

Relational databases  Traditional queries  Show me total sales of last month  Data mining  Searching for trends and patterns  Predict credit risk of new customers based on income, age

Relational Databases..

Data Warehouses  Traditional queries  Show me total sales of last month/year for each city/country  Data mining  Allow more in-depth analysis

Data Warehouses..

Transactional Databases  Traditional queries  Show me all items purchased by Mr. Busybuying  Data mining  Searching for trends and patterns  Which items sold well together

Advanced data sets and advanced applications  Data streams and sensor data  Network traffic, video surveillance, weather monitoring  Detect intrusions of a computer network  We may like to detect intrusions of a computer network based on the anomaly of message flow, which may be discovered by clustering data streams, dynamic construction of stream models, or comparing the current frequent patterns with that at a certain previous time (Mining example).

Advanced data sets and advanced applications  Time-series data, temporal data, sequence data  Stock exchange data, Web click streams  Trends of changes  the mining of banking data may aid in the scheduling of bank tellers according to the volume of customer traffic (Mining Example)  Stock exchange data can be mined to uncover trends that could help you plan investment strategies (e.g., when is the best time to purchase AllElectronics stock?) (Mining Example).

Advanced data sets and advanced applications  Spatial data and spatiotemporal data  Geographic data, CAD databases  Characteristics of houses located near parks  describe the change in trend of metropolitan poverty rates based on city distances from major highways (Spatial mining example)  we may be able to group the trends of moving objects and identify some strangely moving vehicles (spatiotemporal mining example).

Advanced data sets and advanced applications.. Text databases  Keyword associations  By mining text data, one may uncover general and concise descriptions of the text documents, keyword or content associations(Text Mining example) The World-Wide Web  Page clustering and classification Object- Relational databases

Advanced data sets and advanced applications.. Heterogeneous databases and legacy databases  Data mining techniques may provide an interesting solution to the information exchange problem by performing statistical data distribution and correlation analysis, and transforming the given data into higher, more generalized, conceptual levels Multimedia database