1. DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm
Marc Moonen
Dept. E.E./ESAT, KU Leuven

2. Part-III : Optimal & Adaptive Filters
Chapter-11
: Optimal & Adaptive Filters - Intro
General Set-Up
Applications
Optimal (Wiener) Filters
: Least Squares & Recursive Least Squares Estimation
Least Squares Estimation
Recursive Least Squares (RLS) Estimation
Square-Root Algorithms
: Least Means Squares (LMS) Algorithm
LMS/NLMS : Stochastic Gradient Algorithms
LMS analysis
LMS Family
: Fast Recursive Least Squares Algorithms
: Kalman Filtering
Chapter-12
Chapter-13
Chapter-14
Chapter-15

3. Least Mean Squares (LMS) Algorithm

4. Least Mean Squares (LMS) Algorithm

5. Least Mean Squares (LMS) Algorithm
(Widrow 1965 !!)

6. Least Mean Squares (LMS) Algorithm
Bernard Widrow

7. Least Mean Squares (LMS) Algorithm

8. Least Mean Squares (LMS) Algorithm

9. Least Mean Squares (LMS) Algorithm
 large λ_max implies
a small stepsize

10. Least Mean Squares (LMS) Algorithm

11. Least Mean Squares (LMS) Algorithm
error vector projected onto eigenvectors
initial error vector projected onto eigenvectors
(=projection on i-th eigenvector)
small λ_i implies slow convergence
λ_min <<λ_max (hence small μ) implies *very* slow convergence

12. Least Mean Squares (LMS) Algorithm

13. Least Mean Squares (LMS) Algorithm

14. Least Mean Squares (LMS) Algorithm

15. Least Mean Squares (LMS) Algorithm

16. Least Mean Squares (LMS) Algorithm

17. Least Mean Squares (LMS) Algorithm

18. Least Mean Squares (LMS) Algorithm

19. Least Mean Squares (LMS) Algorithm

20. Least Mean Squares (LMS) Algorithm

21. Least Mean Squares (LMS) Algorithm

22. Least Mean Squares (LMS) Algorithm

23. Least Mean Squares (LMS) Algorithm

24. Least Mean Squares (LMS) Algorithm

25. Least Mean Squares (LMS) Algorithm

26. Least Mean Squares (LMS) Algorithm

27. Least Mean Squares (LMS) Algorithm

28. Least Mean Squares (LMS) Algorithm

29. Least Mean Squares (LMS) Algorithm

30. Least Mean Squares (LMS) Algorithm

31. Least Mean Squares (LMS) Algorithm

32. Least Mean Squares (LMS) Algorithm

33. Least Mean Squares (LMS) Algorithm

34. Least Mean Squares (LMS) Algorithm