Neural Network Intro Slides

Slides:

Advertisements

Similar presentations

Artificial Intelligence 12. Two Layer ANNs

Advertisements

Multi-Layer Perceptron (MLP)

Artificial Neural Networks (1)

Ch. Eick: More on Machine Learning & Neural Networks Different Forms of Learning: –Learning agent receives feedback with respect to its actions (e.g. using.

Perceptron Learning Rule

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

1 Neural networks. Neural networks are made up of many artificial neurons. Each input into the neuron has its own weight associated with it illustrated.

G5BAIM Artificial Intelligence Methods Graham Kendall Neural Networks.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Neural NetworksNN 11 Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Connectionist models. Connectionist Models Motivated by Brain rather than Mind –A large number of very simple processing elements –A large number of weighted.

1 Pendahuluan Pertemuan 1 Matakuliah: T0293/Neuro Computing Tahun: 2005.

Introduction to Neural Networks John Paxton Montana State University Summer 2003.

September 28, 2010Neural Networks Lecture 7: Perceptron Modifications 1 Adaline Schematic Adjust weights i1i1i1i1 i2i2i2i2 inininin …  w 0 + w 1 i 1 +

Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:

Introduction to Neural Networks CMSC475/675

1 Introduction to Artificial Neural Networks Andrew L. Nelson Visiting Research Faculty University of South Florida.

CS-485: Capstone in Computer Science Artificial Neural Networks and their application in Intelligent Image Processing Spring

Neural Networks. Plan Perceptron  Linear discriminant Associative memories  Hopfield networks  Chaotic networks Multilayer perceptron  Backpropagation.

Artificial Neural Nets and AI Connectionism Sub symbolic reasoning.

Introduction to Neural Networks. Neural Networks in the Brain Human brain “computes” in an entirely different way from conventional digital computers.

AI History, Philosophical Foundations Part 2. Some highlights from early history of AI Gödel’s theorem: 1930 Turing machines: 1936 McCulloch and Pitts.

Introduction to Artificial Intelligence (G51IAI) Dr Rong Qu Neural Networks.

ADVANCED PERCEPTRON LEARNING David Kauchak CS 451 – Fall 2013.

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

Neural Network Basics Anns are analytical systems that address problems whose solutions have not been explicitly formulated Structure in which multiple.

Back-Propagation Algorithm AN INTRODUCTION TO LEARNING INTERNAL REPRESENTATIONS BY ERROR PROPAGATION Presented by: Kunal Parmar UHID:

Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22

Artificial Neural Networks Chapter 4 Perceptron Gradient Descent Multilayer Networks Backpropagation Algorithm 1.

Introduction to Deep Learning

IE 585 History of Neural Networks & Introduction to Simple Learning Rules.

Introduction to Neural Networks Jianfeng Feng School of Cognitive and Computing Sciences Spring 2001.

COSC 4426 AJ Boulay Julia Johnson Artificial Neural Networks: Introduction to Soft Computing (Textbook)

Artificial Intelligence Methods Neural Networks Lecture 1 Rakesh K. Bissoondeeal Rakesh K.

Perceptrons Michael J. Watts

Previous Lecture Perceptron W  t+1  W  t  t  d(t) - sign (w(t)  x)] x Adaline W  t+1  W  t  t  d(t) - f(w(t)  x)] f’ x Gradient.

Neural Networks Si Wu Dept. of Informatics PEV III 5c7 Spring 2008.

Lecture 12. Outline of Rule-Based Classification 1. Overview of ANN 2. Basic Feedforward ANN 3. Linear Perceptron Algorithm 4. Nonlinear and Multilayer.

Computational Intelligence Semester 2 Neural Networks Lecture 2 out of 4.

Artificial Neural Networks By: Steve Kidos. Outline Artificial Neural Networks: An Introduction Frank Rosenblatt’s Perceptron Multi-layer Perceptron Dot.

Xintao Wu University of Arkansas Introduction to Deep Learning 1.

1 Introduction to Neural Networks Recurrent Neural Networks.

1 Neural Networks MUMT 611 Philippe Zaborowski April 2005.

INTRODUCTION TO NEURAL NETWORKS 2 A new sort of computer What are (everyday) computer systems good at... and not so good at? Good at..Not so good at..

Information Processing

Neural Networks: An Introduction and Overview

Neural networks.

Introduction to Neural Networks

CSCI 5922 Neural Networks and Deep Learning: Introduction 2

Multilayer Perceptrons

Joost N. Kok Universiteit Leiden

Neural Networks CS 446 Machine Learning.

CSE P573 Applications of Artificial Intelligence Neural Networks

Intelligent Leaning -- A Brief Introduction to Artificial Neural Networks Chiung-Yao Fang.

Synaptic DynamicsII : Supervised Learning

CSE (c) S. Tanimoto, 2004 Neural Networks

Intelligent Leaning -- A Brief Introduction to Artificial Neural Networks Chiung-Yao Fang.

CSE 573 Introduction to Artificial Intelligence Neural Networks

G5AIAI Introduction to AI

CSE (c) S. Tanimoto, 2001 Neural Networks

Slides adapted from Geoffrey Hinton, Yann Le Cun, Yoshua Bengio

CSE (c) S. Tanimoto, 2002 Neural Networks

Artificial Intelligence 12. Two Layer ANNs

Deep Learning Authors: Yann LeCun, Yoshua Bengio, Geoffrey Hinton

CSE (c) S. Tanimoto, 2007 Neural Nets

Neural Networks: An Introduction and Overview

David Kauchak CS158 – Spring 2019

Presentation transcript:

Neural Network Intro Slides Michael Mozer Spring 2015

A Brief History Of Neural Networks 1962 Frank Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms Perceptron can learn anything you can program it to do.

A Brief History Of Neural Networks 1969 Minsky & Papert, Perceptrons: An introduction to computational geometry There are many things a perceptron can’t in principle learn to do Early computational complexity analysis: how does learning time scale with size of problem? how does network size scale with problem? How much information does each weight need to represent? Are there classes of functions that can or cannot be computed by perceptrons of a certain architecture? e.g., translation invariant pattern recognition

A Brief History Of Neural Networks 1970-1985 Attempts to develop symbolic rule discovery algorithms 1986 Rumelhart, Hinton, & Williams, Back propagation Overcame many of the Minsky & Papert objections Neural nets popular in cog sci and AI circa 1990

A Brief History Of Neural Networks 1990-2005 Bayesian approaches take the best ideas from neural networks – statistical computing, statistical learning Support-Vector Machines convergence proofs (unlike neural nets) A few old timers keep playing with neural nets Hinton, LeCun, Bengio, O’Reilly Neural nets banished from NIPS!

A Brief History Of Neural Networks 2005-2010 Attempts to resurrect neural nets with unsupervised pretraining probabilistic neural nets alternative learning rules

A Brief History Of Neural Networks 2010-present Most of the alternative techniques discarded in favor of 1980’s style neural nets with lots more training data lots more computing cycles a few important tricks that improve training and generalization (mostly from Hinton)

2013

Key Features of Cortical Computation Neurons are slow (10–3 – 10–2 propagation time) Large number of neurons (1010 – 1011) No central controller (CPU) Neurons receive input from a large number of other neurons (104 fan-in and fan-out of cortical pyramidal cells) Communication via excitation and inhibition Statistical decision making (neurons that single-handedly turn on/off other neurons are rare) Learning involves modifying coupling strengths (the tendency of one cell to excite/inhibit another) Neural hardware is dedicated to particular tasks (vs. conventional computer memory) Information is conveyed by mean firing rate of neuron, a.k.a. activation

Conventional computers One very smart CPU Lots of extremely dumb memory cells Brains, connectionist computers No CPU Lots of slightly smart memory cells

Modeling Individual Neurons

Modeling Individual Neurons threshold

Computation With A Binary Threshold Unit = 1 if net > 0

Computation With A Binary Threshold Unit

Feedforward Architectures

Recurrent Architectures

Supervised Learning In Neural Networks