Neural Networks Vladimir Pleskonjić 3188/2015. 2/20 Vladimir Pleskonjić General Feedforward neural networks Inputs are numeric features Outputs are in.

Slides:

Advertisements

Similar presentations

Multi-Layer Perceptron (MLP)

Advertisements

A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.

NEURAL NETWORKS Backpropagation Algorithm

EE 690 Design of Embodied Intelligence

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

Artificial Neural Networks

Lecture 14 – Neural Networks

Supervised learning 1.Early learning algorithms 2.First order gradient methods 3.Second order gradient methods.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Connectionist models. Connectionist Models Motivated by Brain rather than Mind –A large number of very simple processing elements –A large number of weighted.

September 30, 2010Neural Networks Lecture 8: Backpropagation Learning 1 Sigmoidal Neurons In backpropagation networks, we typically choose  = 1 and 

Artificial Neural Networks Artificial Neural Networks are (among other things) another technique for supervised learning k-Nearest Neighbor Decision Tree.

Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Back-Propagation Algorithm

Neural Networks Chapter Feed-Forward Neural Networks.

Artificial Neural Networks

CHAPTER 11 Back-Propagation Ming-Feng Yeh.

CS 4700: Foundations of Artificial Intelligence

Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.

Hazırlayan NEURAL NETWORKS Radial Basis Function Networks II PROF. DR. YUSUF OYSAL.

Artificial Neural Networks

Classification Part 3: Artificial Neural Networks

Artificial Neural Networks (ANN). Output Y is 1 if at least two of the three inputs are equal to 1.

Multiple-Layer Networks and Backpropagation Algorithms

Integrating Neural Network and Genetic Algorithm to Solve Function Approximation Combined with Optimization Problem Term presentation for CSC7333 Machine.

1 Artificial Neural Networks Sanun Srisuk EECP0720 Expert Systems – Artificial Neural Networks.

Machine Learning Chapter 4. Artificial Neural Networks

11 CSE 4705 Artificial Intelligence Jinbo Bi Department of Computer Science & Engineering

Appendix B: An Example of Back-propagation algorithm

Backpropagation An efficient way to compute the gradient Hung-yi Lee.

Back-Propagation MLP Neural Network Optimizer ECE 539 Andrew Beckwith.

NEURAL NETWORKS FOR DATA MINING

Classification / Regression Neural Networks 2

Artificial Neural Networks. The Brain How do brains work? How do human brains differ from that of other animals? Can we base models of artificial intelligence.

Multi-Layer Perceptron

Non-Bayes classifiers. Linear discriminants, neural networks.

11 1 Backpropagation Multilayer Perceptron R – S 1 – S 2 – S 3 Network.

Neural Networks and Backpropagation Sebastian Thrun , Fall 2000.

ADALINE (ADAptive LInear NEuron) Network and

Introduction to Neural Networks Introduction to Neural Networks Applied to OCR and Speech Recognition An actual neuron A crude model of a neuron Computational.

Chapter 8: Adaptive Networks

Hazırlayan NEURAL NETWORKS Backpropagation Network PROF. DR. YUSUF OYSAL.

Previous Lecture Perceptron W  t+1  W  t  t  d(t) - sign (w(t)  x)] x Adaline W  t+1  W  t  t  d(t) - f(w(t)  x)] f’ x Gradient.

Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer.

Intro. ANN & Fuzzy Systems Lecture 11. MLP (III): Back-Propagation.

Multiple-Layer Networks and Backpropagation Algorithms

The Gradient Descent Algorithm

第 3 章神经网络.

CSE 473 Introduction to Artificial Intelligence Neural Networks

Neural Networks CS 446 Machine Learning.

Classification with Perceptrons Reading:

CSE P573 Applications of Artificial Intelligence Neural Networks

Classification / Regression Neural Networks 2

CSE 473 Introduction to Artificial Intelligence Neural Networks

Prof. Carolina Ruiz Department of Computer Science

Machine Learning Today: Reading: Maria Florina Balcan

CSC 578 Neural Networks and Deep Learning

Artificial Neural Network & Backpropagation Algorithm

Synaptic DynamicsII : Supervised Learning

CSE 573 Introduction to Artificial Intelligence Neural Networks

network of simple neuron-like computing elements

Neural Network - 2 Mayank Vatsa

Backpropagation.

Neural networks (1) Traditional multi-layer perceptrons

Artificial Intelligence 10. Neural Networks

Backpropagation David Kauchak CS159 – Fall 2019.

Backpropagation.

Prof. Carolina Ruiz Department of Computer Science

Presentation transcript:

Neural Networks Vladimir Pleskonjić 3188/2015

2/20 Vladimir Pleskonjić General Feedforward neural networks Inputs are numeric features Outputs are in range (0, 1) Neural network

3/20 Vladimir Pleskonjić Perceptron Building blocks Numeric inputs Output in (0, 1) Bias (always equal to +1) Perceptron

4/20 Vladimir Pleskonjić Perceptron Inputs are summed (bias included) weighted by perceived importance Θ Output is sigmoid function of this sum Sigmoid function

5/20 Vladimir Pleskonjić Architecture Parallel perceptrons form one neural layer Serial layers form the neural network Multiple hidden layers are allowed Neural layers

6/20 Vladimir Pleskonjić Feedforward Calculating output is called feedforward Layer by layer Outputs are confidences for their class Interpreted by the user

7/20 Vladimir Pleskonjić Training Weights Θ are the soul of our network Train examples Weights are configured to minimize the error function

8/20 Vladimir Pleskonjić Error function Error per output Error per train example Total error Error for single output

9/20 Vladimir Pleskonjić Backpropagation Partial derivatives of error function with respect to each weight in Θ Derivative rules and formulae

10/20 Vladimir Pleskonjić Optimization Gradient descent Learning rate Weights should be initialized with small random values Descent

11/20 Vladimir Pleskonjić Regularization Overfitting Addition to error function Sum of squares of weights Regularization coefficient Not applied on links connecting to biases

12/20 Vladimir Pleskonjić Stochastic gradient descent Train cycles Batch (all at once) Stochastic (one by one) Mini-batch (in groups)

13/20 Vladimir Pleskonjić Putting it together Design architecture Randomize weights Train cycle: backpropagation + gradient descent Repeat train cycles a number of times Feedforward to use Interpret output confidences

14/20 Vladimir Pleskonjić Implementation C++ Matrix class CUDA used for some of the operations NeuralNetwork class

15/20 Vladimir Pleskonjić Code: feedforward

16/20 Vladimir Pleskonjić Code: gradient descent

17/20 Vladimir Pleskonjić Pseudo code: training

18/20 Vladimir Pleskonjić Results MNIST database My configuration Train accuracy: 95.3% Test accuracy: 95.1%

19/20 Vladimir Pleskonjić Performance C++ [ms]C++ and CUDA [ms] 1% TRAIN % TEST % TRAIN % TEST % TRAIN ?? 100% TEST Execution time comparison The aim of this project was not to achieve the optimal performance, but to reach a better understanding of the algorithm and its implementation.

20/20 Vladimir Pleskonjić Questions? Thank you!