Computer Science Department Learning Processes CS679 Lecture Note by Jin Hyung Kim Computer Science Department KAIST

Learning Learning is a process by which free parameters of NN are adapted thru stimulation from environment Sequence of Events stimulated by an environment undergoes changes in its free parameters responds in a new way to the environment Learning Algorithm prescribed steps of process to make a system learn ways to adjust synaptic weight of a neuron No unique learning algorithms - kit of tools The Chapter covers five learning rules, learning paradigms, issues of learning task probabilistic and statistical aspect of learning

Error Correction Learning(I) Error signal, ek(n) ek(n) = dk(n) - yk(n) where n denotes time step Error signal activates a control mechanism for corrective adjustment of synaptic weights Mininizing a cost function, E(n), or index of performance Also called instantaneous value of error energy step-by-step adjustment until system reaches steady state; synaptic weights are stabilized Also called deltra rule, Widrow-Hoff rule

Error Correction Learning(II) wkj(n) = ek(n)xj(n)  : rate of learning; learning-rate parameter wkj(n+1) = wkj(n) + wkj(n) wkj(n) = Z-1[wkj(n+1) ] Z-1 is unit-delay operator adjustment is proportioned to the product of error signal and input signal error-correction learning is local The learning rate  determines the stability or convergence

Memory-based Learning Past experiences are stored in memory of correctly classified input-output examples retrieve and analyze “local neighborhood” Essential Ingredient Criterion used for defining local neighbor Learning rule applied to the training examples Nearest Neighbor Rule (NNR) the vector X’n  { X1, X2, …,XN } is the nearest neighbor of Xtest if X’n is the class of Xtest

Nearest Neighbor Rule Cover and Hart K-nearest Neighbor rule Examples are independent and identically distributed The sample size N is infinitely large then, error(NNR) < 2 * error(Bayesian rule) Half of the information is contained in the Nearest Neighbor K-nearest Neighbor rule variant of NNR Select k-nearest neighbors of Xtest and use a majority vote act like averaging device discriminate against outlier Radial-basis function network is a memory-based classifier

Hebbian Learning If two neurons of a connection are activated simultaneously (synchronously), then its strength is increased asynchronously, then the strength is weakened or eliminated Hebbian synapse time dependent depend on exact time of occurrence of two signals local locally available information is used interactive mechanism learning is done by two signal interaction conjunctional or correlational mechanism cooccurrence of two signals Hebbian learning is found in Hippocampus presynaptic & postsynaptic signals

Math Model of Hebbian Modification wkj(n) = F(yk(n), xj(n)) Hebb’s hypothesis wkj(n) = yk(n)xj(n) where  is rate of learning also called activity product rule repeated application of xj leads to exponential growth Covariance hypothesis wkj(n) = (xj - x)(yk - y) 1. wkj is enhanced if xj > x and yk > y 2. wkj is depressed if (xj > x and yk < y ) or (xj < x and yk > y ) where x and y are time-average

Competitive Learning Output neurons of NN compete to became active Only single neuron is active at any one time salient feature for pattern classification Basic Elements A set of neurons that are all same except synaptic weight distribution responds differently to a given set of input pattern A Limits on the strength of each neuron A mechanism to compete to respond to a given input winner-takes-all Neurons learn to respond specialized conditions become feature detectors

Competitive NN Feedforward connection is excitatory Lateral is inhibitory - lateral inhibition layer of source Input Single layer of output neurons

Competitive Learning Output signal if vk > vj for all j, j  k otherwise Wkj denotes the synaptic weight Competitive Learning Rule If neuron does not respond to a particular input, no learning takes place

Competitive Learning x has some constant Euclidean length and perform clustering thru competitive learning

Boltzman Learning Rooted from statistical mechanics Boltzman Machine : NN on the basis of Boltzman learning The neurons constitute a recurrent structure operate in binary manner: “on”: +1 and “off”: -1 Visible neurons and hidden neurons energy function means no self feedback Goal of Boltzman Learning is to maximize likelihood function set of synaptic weights is called a model of the environment if it leads the same probability distribution of the states of visible units j  k j  k

Boltzman Machine Operation choosing a neuron at random, k, then flip the state of the neuron from state xk to state -xk with probability where is energy change resulting from such a flip If this rule applied repeatedly, the machine reaches thermal equilibrium. Two modes of operation Clamped condition : visible neurons are clamped onto specific states Free-running condition: all neurons are allowed to operate freely

Boltzman Learning Rule Let denote the correlation between the states of neurons j and k with its clamped condition Let denote the correlation between the states of neurons j and k with its free-running condition Boltzman Learning Rule (Hinton and Sejnowski 86) where  is a learning-rate

Credit-Assignment Problem Problems of assigning credit or blame for overall outcomes to each of internal decisions Two sub-problems assignment of credit to actions temporal credit assignment problem : time instance of action many actions are taken; which action is responsible the outcome assignment of credit to internal decisions structural credit assignment problem : internal structure of action multiple components are contributed; which one do best CAP occurs in Error-correction learning in multiple layer feed-forward NN How do we credit or blame for the actions of hidden neurons ?

Learning with Teacher Supervised learning Teacher has knowledge of environment to learn input and desired output pairs are given as a training set Parameters are adjusted based on error signal step-by-step System performance measure mean-square-error sum of squared errors over the training sample visualized as error surface with parameter as coordinates Move toward to a minimum point of error surface may not be a global minimum use gradient of error surface - direction of steepest descent Good for pattern recognition and function approximation

Learning without Teacher Reinforcement learning No teacher to provide direct (desired) response at each step example : good/bad, win/loose must solve temporal credit assignment problem since critics may be given at the time of final output Environment Critics Learning Systems Primary reinforcement Heuristic reinforcement

Unsupervised Learning Self-organized learning No external teacher or critics Task-independent measure of quality is required to learn Network parameters are optimized with respect to the measure competitive learning rule is a case of unsupervised learning

Learning Tasks Pattern Association Pattern Recognition Function Approximation Control Filtering Beamforming

Pattern Association Associative memory is distributed memory that learns by association predominant features of human memory xk  yk, k=1, 2,…, q Storage capacity, q Storage phase and Recall phase NN is required to store a set of patterns by repeatedly presenting partial description or distorted pattern is used to retrieve the particular pattern xk is act as a stimulus that determines location of memorized pattern Autoassociation : when xk = yk, Hetroassociation : when xk  yk, Have q as large as possible yet recall correctly

Pattern Recognition Process whereby a received pattern is assigned to one of prescribed classes (categories) Pattern recognition by NN is statistical in nature pattern is a point in a multidimensional decision space decision boundaries - region associated with a class decision boundaries are determined by training Feature Extraction Classification Input pattern Outout class Feature vector unsupervised supervised r-D m-D q-D

Function Approximation Given set of labeled examples drawn from , find which approximates unknown function s.t. Supervised learning is good for this task System identification construct an inverse system di Unknown system NN model  ei input yi

Control Feedback control system search of Jacobian matrix for error correction learning indirect learning using input-output measurement on the plant, construct NN model the NN model is used to estimate Jacobian direct learning Controller Plant  Reference signal d y e u

Filtering Extract information from a set of noisy data information processing tasks filtering using data measured up to and including time n smoothing using data measured up to and after time n prediction predict at n+n0 ( n0 > 0) using data measured up to time n Cocktail party problem focus on a speaker in the noisy environment believed that preattentive, preconscious analysis involved

Blind source separation x1(n) Unknown mixer A Demixer W u1(n) y1(n) ... ... ... um(n) ym(n) xm(n) Unknown environment

Memory Relatively enduring neural alterations induced by interaction with environment - neurobiological definition accessible to influence future behavior activity pattern is stored by learning process memory and learning are connected Short-term memory compilation of knowledge representing current state of environment Long-term memory knowledge stored for a long time or permanently

Associative memory Memory is distributed stimulus pattern and response pattern consist of data vectors information is stored as spatial patterns of neural activities information of stimulus pattern determines not only stage location but also address for its retrieval although neurons is not reliable, memory is there may be interaction between patterns stored. (not a simple storage of patterns) There is possibility of error in recall process

Correlation Matrix Memory Association of key vector xk with memorized vector yk yk = W(k)xk, k = 1, 2, 3, …, q Total experience Adding W(k) to Mk-1 loses distinct identity of mixture of contributions that form Mk but information about stimuli is not lost

Correlation Matrix memory Learning : Estimate of memory matrix M called Outer Product Rule generalization of Hebb’s postulate of learning

Recall Normalize to have unit matrix defined as Vj Noise Vector normalized to have unit matrix

Orthogonal set orthogonal set Storage Capacity rank : # of independent columns limited by dimensionality of input

Error of Associative Memory Real-life is Neither orthogonal nor highly separated Lower bound  if  is large, the memory may fail to distinguish memory act as having patterns of never seen before termed animal logic

Adaptation Space and time : fundamental dimension Stationary system spatiotemporal nature of learning Stationary system learned parameters are frozen to be used later Nonstationary system continually adapt its parameters to variations in the incoming signal in a real-time fashion continuous learning; learning-on-the-fly Psudostationary consider stationary for a short time window speech signal : 10~30 msec weather forcasting : period of minutes

Statistical Nature of Learning Process Training sample denoted by functional relationship between X and D : deterministic function of Random variable X expectation error random variable called regressive model Mean value is zero error is uncorrelated (principles of orthogonality)

Minimizing Cost Function E is averaging operator natural measure of the effectiveness since

Bias/Variance Dilemma represents inability of the NN to approximate the regression function variance inadequacy of the information contained in the training set  we can reduce both of bias and variance only with Large training samples purposely introduce “harmless” bias to reduce variance designed bias - constrained network takes prior knowledge

Statistical Learning Theory Address fundamental issue of controlling generalization ability Principles of empirical risk minimization empirical risk function uniform convergence As training progress, likelihood of those approximate functions F(xi, w), consistent with training data  is increased

Vapnik-Chervonenkis Dimension Measure the capacity or expressive power of the family of classification functions realized by the learning machine VC dimension, VCdim(), of an ensemble of dichotomies  is the cardinality of the largest set L that is shattered by  maximum number of training examples that can be shattered by the machine() without error example : 4 points in general position in 2D space # of dichotomies ((L)): 16 Vcdim(), max N where (L) = 2N : 4 evaluation of Vcdim(.) is defficult task, but its bounds are often tractable feedforward network with a threshold function : O(W logW) feedforward network with the sigmoid function : O(W2) feedforward network with pure linear function : O(W)

Confidence Interval A finite VCdim is a necessary and sufficient condition for the uniform convergence of the principle of empirical risk minimization Training Error vs Generalization Error

Guaranteed risk Guranteed risk (bound on generation error error) Training error VC dimension, h overdetermined underdetermined

Probably Approximately Correct Learning Model Goal I of PAC learning is to ensure that vtrain is usually small prob. of difference btn trained hypothesis and concept to learn Parameters size of training sample, N error parameter, , allowed in approximation of the target concept confidence parameters, , likelihood of approximation How many random example is needed to learn unknown concept within given VCdim h,  and  with constant K For H hypothesis