Neural Nets for Data Mining

Slides:



Advertisements
Similar presentations
1 Data Mining: and Knowledge Acquizition — Chapter 5 — BIS /2014 Summer.
Advertisements

CSE 634 Data Mining Techniques
Data Mining Classification: Alternative Techniques
Intelligent Environments1 Computer Science and Engineering University of Texas at Arlington.
Machine Learning Neural Networks
Neural Network Oleh Danny Manongga
Artificial Intelligence (CS 461D)
Neural NetworksNN 11 Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22
Decision Support Systems
1 MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING By Kaan Tariman M.S. in Computer Science CSCI 8810 Course Project.
Data Mining with Neural Networks (HK: Chapter 7.5)
LOGO Classification III Lecturer: Dr. Bo Yuan
Gini Index (IBM IntelligentMiner)
Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.
Data Mining.
CHAPTER 12 ADVANCED INTELLIGENT SYSTEMS © 2005 Prentice Hall, Decision Support Systems and Intelligent Systems, 7th Edition, Turban, Aronson, and Liang.
Machine Learning. Learning agent Any other agent.
MSE 2400 EaLiCaRA Spring 2015 Dr. Tom Way
Artificial Intelligence Lecture No. 28 Dr. Asad Ali Safi ​ Assistant Professor, Department of Computer Science, COMSATS Institute of Information Technology.
Artificial Neural Networks (ANN). Output Y is 1 if at least two of the three inputs are equal to 1.
Using Neural Networks in Database Mining Tino Jimenez CS157B MW 9-10:15 February 19, 2009.
Neural NetworksNN 11 Neural netwoks thanks to: Basics of neural network theory and practice for supervised and unsupervised.
Artificial Neural Nets and AI Connectionism Sub symbolic reasoning.
IE 585 Introduction to Neural Networks. 2 Modeling Continuum Unarticulated Wisdom Articulated Qualitative Models Theoretic (First Principles) Models Empirical.
Chapter 9 Neural Network.
Neural Networks AI – Week 23 Sub-symbolic AI Multi-Layer Neural Networks Lee McCluskey, room 3/10
Chapter 3 Neural Network Xiu-jun GONG (Ph. D) School of Computer Science and Technology, Tianjin University
Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.
NEURAL NETWORKS FOR DATA MINING
Introduction to machine learning and data mining 1 iCSC2014, Juan López González, University of Oviedo Introduction to machine learning Juan López González.
LINEAR CLASSIFICATION. Biological inspirations  Some numbers…  The human brain contains about 10 billion nerve cells ( neurons )  Each neuron is connected.
Artificial Neural Networks. The Brain How do brains work? How do human brains differ from that of other animals? Can we base models of artificial intelligence.
Some working definitions…. ‘Data Mining’ and ‘Knowledge Discovery in Databases’ (KDD) are used interchangeably Data mining = –the discovery of interesting,
1 Introduction to Neural Networks And Their Applications.
Chapter 6: Artificial Neural Networks for Data Mining
Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22
Dr.Abeer Mahmoud ARTIFICIAL INTELLIGENCE (CS 461D) Dr. Abeer Mahmoud Computer science Department Princess Nora University Faculty of Computer & Information.
Data Mining and Decision Support
Perceptrons Michael J. Watts
Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.
Artificial Neural Networks for Data Mining. Copyright © 2011 Pearson Education, Inc. Publishing as Prentice Hall 6-2 Learning Objectives Understand the.
“Principles of Soft Computing, 2 nd Edition” by S.N. Sivanandam & SN Deepa Copyright  2011 Wiley India Pvt. Ltd. All rights reserved. CHAPTER 2 ARTIFICIAL.
Pattern Recognition. What is Pattern Recognition? Pattern recognition is a sub-topic of machine learning. PR is the science that concerns the description.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining.
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining.
Supervised Learning – Network is presented with the input and the desired output. – Uses a set of inputs for which the desired outputs results / classes.
Network Management Lecture 13. MACHINE LEARNING TECHNIQUES 2 Dr. Atiq Ahmed Université de Balouchistan.
Data Mining: Concepts and Techniques1 Prediction Prediction vs. classification Classification predicts categorical class label Prediction predicts continuous-valued.
A Presentation on Adaptive Neuro-Fuzzy Inference System using Particle Swarm Optimization and it’s Application By Sumanta Kundu (En.R.No.
Artificial Neural Networks This is lecture 15 of the module `Biologically Inspired Computing’ An introduction to Artificial Neural Networks.
INTRODUCTION TO NEURAL NETWORKS 2 A new sort of computer What are (everyday) computer systems good at... and not so good at? Good at..Not so good at..
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining.
Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.
Multiple-Layer Networks and Backpropagation Algorithms
Neural Network Architecture Session 2
Fall 2004 Perceptron CS478 - Machine Learning.
Other Classification Models: Neural Network
Learning with Perceptrons and Neural Networks
Learning in Neural Networks
Data Mining, Neural Network and Genetic Programming
Artificial Intelligence (CS 370D)
Announcements HW4 due today (11:59pm) HW5 out today (due 11/17 11:59pm)
with Daniel L. Silver, Ph.D. Christian Frey, BBA April 11-12, 2017
Data Mining with Neural Networks (HK: Chapter 7.5)
Neural Networks Advantages Criticism
Neuro-Computing Lecture 4 Radial Basis Function Network
Artificial Intelligence Lecture No. 28
MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING
MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING
A task of induction to find patterns
Presentation transcript:

Neural Nets for Data Mining 2014/5/9 Neural Nets for Data Mining CISC 6930  Data Mining School of Information Science and Engineering, Central South University

Outline Neural Networks: Background Neural Network Classifier ANN Architecture Strength and Weakness of ANN Applications

Nearest Neighbor Classifier Data Mining Common data mining tasks Classification [Predictive] Clustering [Descriptive] Association Rule Discovery [Descriptive] Sequential Pattern Discovery [Descriptive] Regression [Predictive] Deviation Detection [Predictive] Classifiers Decision Trees Rule Approaches Logical statements (ILP) Bayesian Classifiers Nearest Neighbor Learning Neural Networks Discriminant Analysis Support Vector Machines Logistic regression Artificial Neural Networks Genetic Classifiers ...

Learning Objectives Learn the step-by-step process of how to use NN for data mining Understand a variety of applications of NN, solving problem types of Classification Regression Clustering Predicition

Neural Networks: Background The first learning algorithm came in 1959 (Rosenblatt) who suggested that if a target output value is provided for a single neuron with fixed inputs, one can incrementally change weights to learn to produce these outputs using the perceptron learning rule

Neural Networks: Background What is NN? Biologically motivated approach to machine learning Similarity with biological network Indeed a great example of a good learning system

Neural Networks: Background What is NN? Biologically motivated approach to machine learning Similarity with biological network Fundamental processing elements of a neural network is a neuron A human brain has 100 billion neurons An ant brain has 250,000 neurons Synapses, the basis of learning and memory

Neural Networks: Background NNs is a set of connected INPUT/OUTPUT UNITS, where each connection has a WEIGHT associated with it. NNs learning is also called CONNECTIONIST learning due to the connections between units. It is a case of SUPERVISED, INDUCTIVE or CLASSIFICATION learning.

Neural Networks: Background Biology Analogy

Outline Neural Networks: Background Neural Network Classifier ANN Architecture Strength and Weakness of ANN Applications

Neural Network Classifier Input: Classification data It contains classification attribute Data is divided, as in any classification problem. [Training data and Testing data] All data must be normalized i.e. all values of attributes in the database are changed to contain values in the internal [0,1] or[-1,1] Neural Network can work with data in the range of (0,1) or (-1,1) Basic normalization techniques for data classification Max-Min normalization Decimal Scaling normalization

Data Normalization Min-max normalization Name Gender Salary A M 87000 B F 73600 C 65000 D 76000 E 56200 Consider employees income range between $56200 to $87000. If this range is normalized to [0, 1], what is the B’s normalized salary?

Data Normalization Decimal Scaling Normalization Normalization by decimal scaling normalizes by moving the decimal point of values of attribute A. Here j is the smallest integer such that max|v’|<1. Example : A – values range from -986 to 917. Max |v| = 986. v = -986 normalize to v’ = -986/1000 = -0.986

One Neuron as a Network w1 = 0.5 and w2 = 0.5 An artificial neuron is a mathematical function conceived as a model of biological neurons. Artificial neurons are the constitutive units in an artificial neural network. Here, x1 and x2 are normalized attribute value of data. y is the output of the neuron , i.e the class label. Value of x1 is multiplied by a weight w1 and values of x2 is multiplied by a weight w2. Given that w1 = 0.5 and w2 = 0.5 Say value of x1 is 0.3 and value of x2 is 0.8, So, weighted sum is : sum= w1 x x1 + w2 x x2 = 0.5 x 0.3 + 0.5 x 0.8 = 0.55

One Neuron as a Network An artificial neuron is a mathematical function conceived as a model of biological neurons. Artificial neurons are the constitutive weight units in an artificial neural network (ANN).

One Neuron as a Network The neuron receives the weighted sum as input and calculates the output as a function of input as follows : y = f(x) , where f(x) is defined as f(x) = 0 { when x< 0.5 } f(x) = 1 { when x >= 0.5 } For our ex ample, x ( weighted sum ) is 0.55, so y = 1 , That means corresponding input attribute values are classified in class 1. If for another input values , x = 0.45 , then f(x) = 0, so we could conclude that input values are classified to class 0.

Outline Neural Networks: Background Neural Network Classifier ANN Architecture Strength and Weakness of ANN Applications

ANN Architecture Formally, ANN is specified by: Neuron model ANN is a machine learning approach that models human brain and consists of a number of artificial neurons. Each neuron in ANN receives a number of inputs. An architecture A set of neurons and links connecting neurons. Each link has a weight Neuron tends to have fewer connections than biological neurons. A learning algorithm It used for training the NN by modifying the weights in order An activation function is applied to these inputs which results in activation level of neuron (output value of the neuron). Knowledge about the learning task is given in the form of examples called training examples.

ANN Architecture Formally, ANN is specified by: Neuron model ANN is a machine learning approach that models human brain and consists of a number of artificial neurons. Each neuron in ANN receives a number of inputs, e.g., x1, x2…xn An architecture A set of neurons and links connecting neurons. Each link has a weight A set of links, describing the neuron inputs, with weights W1, W2, …, Wm A learning algorithm It used for training the NN by modifying the weights in order An activation function is applied to these inputs which results in activation level of neuron (output value of the neuron). For limiting the amplitude of the neuron output. Here ‘b’ denotes bias.

How Does the ANN Learn? A neural network learns by determining the relation between the inputs and outputs. By calculating the relative importance of the inputs and outputs the system can determine such relationships. Through trial and error the system compares its results with the expert provided results in the data until it has reached an accuracy level defined by the user. With each trial the weight assigned to the inputs is changed until the desired results are reached.

A Single Layer ANN x0 = +1 x1 v y x2 xm wm We need the bias value to be added to the weighted sum ∑wixi so that we can transform it from the origin. v = ∑wixi + b, here b is the bias Input Attribute values weights Summing function Activation v Output class x1 x2 xm w2 wm W1 w0 x0 = +1 y induced field of the neuron

A Single Layer ANN x0 = +1 x1 v y x2 xm wm We need the bias value to be added to the weighted sum ∑wixi so that we can transform it from the origin. v = ∑wixi + b, here b is the bias Input Attribute values weights Summing function Activation v x1 x2 xm w2 wm W1 w0 x0 = +1 y Output class

Multi-Layer Perceptron Output Class Output nodes Hidden nodes wij weights Input nodes Network is fully connected Input Record : xi

Single Layer vs. Multi Layers Output layer Input layer Input layer Output layer Hidden Layer 3-4-2 Network

Network Training Backpropagation algorithm The ultimate objective of training Obtain a set of weights that makes almost all the tuples in the training data classified correctly Steps Initialize weights with random values Feed the input tuples into the network one by one For each unit Compute the net input to the unit as a linear combination of all the inputs to the unit Compute the output value using the activation function Compute the error Update the weights and the bias

Network Pruning and Rule Extraction Fully connected network will be hard to articulate N input nodes, h hidden nodes and m output nodes lead to h(m+N) weights Pruning Remove some of the links without affecting classification accuracy of the network

Outline Neural Networks: Background Neural Network Classifier ANN Architecture Strength and Weakness of ANN Applications

Strength of ANN ANN has a high tolerance to noisy and incomplete data Massive parallelism allowing for computational efficiency Autonomous learning and generalization Able to deal with (identify/model) highly nonlinear relationships Usually provides better results (prediction and/or clustering) compared to its statistical counterparts

Weakness of ANN Training may take a long time for large datasets; which may require case sampling It is hard to find optimal values for large number of network parameters Optimal design is still an art: requires expertise and extensive experimentation It is hard to handle large number of variables

Outline Neural Networks: Background Neural Network Classifier ANN Architecture Strength and Weakness of ANN Applications

Application-I Handwritten Digit Recognition Face recognition Time series prediction Process identification Process control Optical character recognition

Application-II Forecasting/Market Prediction: finance and banking Manufacturing: quality control, fault diagnosis Medicine: analysis of electrocardiogram data, RNA & DNA sequencing, drug development without animal testing Control: process, robotics

Data Mining Software Supporting ANN PASW (formerly SPSS Clementine) SAS Enterprise Miner Statistica Data Miner, … many more …

Reference Chapter 7.5 Professor Anita Wasilewska’s lecture note, www.cs.vu.nl/~elena/slides03/nn_1light.ppt Xin Yao Evolving Artificial Neural Networks http://www.cs.bham.ac.uk/~xin/papers/published_iproc_sep99.pdf informatics.indiana.edu/larryy/talks/S4.MattI.EANN.ppt www.cs.appstate.edu/~can/classes/5100/Presentations/DataMining1.ppt www.comp.nus.edu.sg/~cs6211/slides/blondie24.ppt www.public.asu.edu/~svadrevu/UMD/ThesisTalk.ppt www.ctrl.cinvestav.mx/~yuw/file/afnn1_nnintro.PPT

Q & A