Jochen Dijrre, Peter Gerstl, Roland Seiffert Presented by Trevor Crum 04/23/2014 *Slides modified from Shamil Mustafayev’s 2013 presentation * 1

Outline  Definition and Paper Overview  Motivation  Methodology  Feature Extraction  Clustering and Categorizing  Some Applications  Comparison with Data Mining  Conclusion & Exam Questions 2

Definition  Text Mining: The discovery by computer of new, previously unknown information, by automatically extracting information from different unstructured textual documents. Also referred to as text data mining, roughly equivalent to text analytics which refers more specifically to problems based in a business settings. 3

Paper Overview  This paper introduced text mining and how it differs from data mining proper.  Focused on the tasks of feature extraction and clustering/categorization  Presented an overview of the tools/methods of IBM’s Intelligent Miner for Text 4

Outline  Definition and Paper Overview  Motivation  Methodology  Feature Extraction  Clustering and Categorizing  Some Applications  Comparison with Data Mining  Conclusion & Exam Questions 5

Motivation  A large portion of a company’s data is unstructured or semi-structured – about 90% in 1999! 6  Letters  s  Phone transcripts  Contracts  Technical documents  Patents  Web pages  Articles

Typical Applications  Summarizing documents  Discovering/monitoring relations among people, places, organizations, etc  Customer profile analysis  Trend analysis  Document summarization  Spam Identification  Public health early warning  Event tracks 7

Outline  Definition and Paper Overview  Motivation  Methodology  Comparison with Data Mining  Feature Extraction  Clustering and Categorizing  Some Applications  Conclusion & Exam Questions 8

Methodology: Challenges  Information is in unstructured textual form  Natural language interpretation is difficult & complex task! (not fully possible) Google and Watson are a step closer  Text mining deals with huge collections of documents Impossible for human examination 9

Google vs Watson  Google justifies the answer by returning the text documents where it found the evidence.  Google finds documents that are most suitable to a given Keyword. 10  Watson tries to understand the semantics behind a given key phrase or question.  Then Watson will use its huge knowledge base to find the correct answer.  Watson uses more AI

Methodology: Two Aspects  Knowledge Discovery Extraction of codified information ○ Feature Extraction Mining proper; determining some structure  Information Distillation Analysis of feature distribution 11

Two Text Mining Approaches  Extraction Extraction of codified information from a single document  Analysis Analysis of the features to detect patterns, trends, and other similarities over whole collections of documents 12

Outline  Definition and Paper Overview  Motivation  Methodology  Feature Extraction  Clustering and Categorizing  Some Applications  Comparison with Data Mining  Conclusion & Exam Questions 13

Feature Extraction  Recognize and classify “significant” vocabulary items from the text  Categories of vocabulary Proper names – Mrs. Albright or Dheli, India Multiword terms – Joint venture, online document Abbreviations – CPU, CEO Relations – Jack Smith-age-42 Other useful things: numerical forms of numbers, percentages, money, dates, and many other 14

Canonical Form Examples  Normalize numbers, money Four = 4, five-hundred dollar = $500  Conversion of date to normal form 8/17/1992 = August  Morphological variants Drive, drove, driven = drive  Proper names and other forms Mr. Johnson, Bob Johnson, The author = Bob Johnson 15

Feature Extraction Approach  Linguistically motivated heuristics  Pattern matching  Limited lexical information (part-of- speech)  Avoid analyzing with too much depth Does not use too much lexical information No in-depth syntactic or semantic analysis 16

IBM Intelligent Miner for Text  IBM introduced Intelligent Miner for Text in 1998  SDK with: Feature extraction, clustering, categorization, and more  Traditional components (search engine, etc) 17

Advantages to IBM’s approach  Processing is very fast (helps when dealing with huge amounts of data)  Heuristics work reasonably well  Generally applicable to any domain 18

Outline  Definition and Paper Overview  Motivation  Methodology  Comparison with Data Mining  Feature Extraction  Clustering and Categorizing  Some Applications  Conclusion & Exam Questions 19

Clustering  Fully automatic process  Documents are grouped according to similarity of their feature vectors  Each cluster is labeled by a listing of the common terms/keywords  Good for getting an overview of a document collection 20

Two Clustering Engines  Hierarchical clustering Orders the clusters into a tree reflecting various levels of similarity  Binary relational clustering Flat clustering Relationships of different strengths between clusters, reflecting similarity 21

Clustering Model 22

Categorization  Assigns documents to preexisting categories  Classes of documents are defined by providing a set of sample documents.  Training phase produces “categorization schema”  Documents can be assigned to more than one category  If confidence is low, document is set aside for human intervention 23

Categorization Model 24

Outline  Definition and Paper Overview  Motivation  Methodology  Feature Extraction  Clustering and Categorizing  Some Applications  Comparison with Data Mining  Conclusion & Exam Questions 25

Applications  Customer Relationship Management application provided by IBM Intelligent Miner for Text called “Customer Relationship Intelligence” or CRI “Help companies better understand what their customers want and what they think about the company itself” 26

Customer Intelligence Process  Take as input body of communications with customer  Cluster the documents to identify issues  Characterize the clusters to identify the conditions for problems  Assign new messages to appropriate clusters 27

Customer Intelligence Usage  Knowledge Discovery Clustering used to create a structure that can be interpreted  Information Distillation Refinement and extension of clustering results ○ Interpreting the results ○ Tuning of the clustering process ○ Selecting meaningful clusters 28

Outline  Definition and Paper Overview  Motivation  Methodology  Feature Extraction  Clustering and Categorizing  Some Applications  Comparison with Data Mining  Conclusion & Exam Questions 29

Comparison with Data Mining  Data mining Discover hidden models. tries to generalize all of the data into a single model. marketing, medicine, health care 30  Text mining Discover hidden facts. tries to understand the details, cross reference between individual instances biosciences, customer profile analysis

Outline  Definition and Paper Overview  Motivation  Methodology  Feature Extraction  Clustering and Categorizing  Some Applications  Comparison with Data Mining  Conclusion & Exam Questions 31

Conclusion  Text mining can be used as an effective business tool that supports Creation of knowledge by preparing and organizing unstructured textual data [Knowledge Discovery] Extraction of relevant information from large amounts of unstructured textual data through automatic pre-selection based on user defined criteria [Information Distillation] 32

Exam Question #1  What are the two aspects of Text Mining when applied to customer complaints? Knowledge Discovery: Discovering a common customer complaint in a large collection of documents containing customer feedback. Information Distillation: Filtering future complaints into pre-defined categories 33

Exam Question #2  How does the procedure for text mining differ from the procedure for data mining? Adds feature extraction phase Infeasible for humans to select features manually The feature vectors are, in general, highly dimensional and sparse 34

Exam Question #3  What are some examples of unstructured textual collections used in Text Mining? Customer letters correspondence Phone transcripts Technical documentation Patents Many others 35

36