1. Widrow-Hoff Learning

2. Outline 1 Introduction 2 ADALINE Network 3 Mean Square Error 4 LMS Algorithm 5 Analysis of Converge 6 Adaptive Filtering

3. Introduction In 1960, Bernard Widrow and his doctoral student Marcian Hoff introduced the ADALINE (ADAptive LInear NEuron)network and LMS(Least Mean Square) algorithm.

4. Perceptron Network Figure: a=hardlim(Wp+b)

5. ADALINE Network Figure: a=purelin(Wp+b)=Wp+b

6. Single ADALINE

7. decision boundary

8. Mean Square Error

9. Mean Square Error(conti.)

11. Error analysis

12. Error analysis(conti.) d = -2h and A = 2R = 0 definite

13. Example 1

14. Example 1(conti.)

16. Approximate Steepest Descent

17. Approximate Gradient

18. Approximate Gradient(conti.)

20. LMS Algorithm

21. LMS Algorithm (conti.)

22. Example 2

23. Example 2(conti.)

27. Analysis of Convergence

28. Analysis of Convergence(conti.)

30. Example 3

31. Perceptron rule V.S. LMS algorithm

32. Perceptron rule V.S. LMS algorithm(conti.)

35. Adaptive Filtering

36. Tapped Delay Line

37. Adaptive Filter

38. Adaptive Noise Cancellation