1. Applications of Filters

2. Running Average of length M f = M -1 [1, 1, 1, … 1] T time, t f t M-1 0 M -1 t0t0

3. Note that average is “delayed” … time, t x t M-1 t0t0 f*xf*x 0 M -1 t0t0 0 1 center Could be ‘re- centered’ during plotting’ … since no future values of x are available

4. MatLab Example M = 30; % length of filter f = (1/M) * ones(M,1); y = conv(f,T); Note MatLab has an built-in convolution function If x has length N and y has length M then x*y has length N+M-1

5. Laguardia Airport Mean Temperature Blue: daily mean temperature data Red: 14 day running average 2 years of data

6. Laguardia Airport Mean Temperature Blue: daily mean temperature data Red: 30 day running average 2 years of data

7. Laguardia Airport Mean Temperature Blue: daily mean temperature data Red: 365 day running average Note “edge effect” 20 years of data

8. Criticism of Running Average: it has sharp edges time, t f t M-1 0 M -1 t0t0 sharp edge The filter can produce rough results, because, as the filter advances in time, an outlier suddenly goes from being inside to outside

9. Gaussian running average time, t f t M-1 0 M -1 t0t0 time, t f t M-1 0 t0t0 Need to discard a bit of future here 67% of area between t 0 and t M-1

10. MatLab Example width = 14; % 67 percent width sigma = 14/2; % standard deviation M = 6*round(sigma/dt); % truncate filter at +/- 3 sigma delay = 3*sigma; % center of filter f = exp( -(t(1:M)-delay).^2 / (2*sigma*sigma) ); amp = sum(f); % normalize to unit amplitude f = f/amp;

11. 14 day Gaussian filter 14 days

12. Laguardia Airport Mean Temperature Blue: daily mean temperature data Red: 14 day Gaussian running average 2 years of data

13. Laguardia Airport Mean Temperature Blue: daily mean temperature data Red: 14 day Gaussian running average “Box car”“Gaussian”

14. Exponential was easy to implement recursively time, t f t M-1 0 M -1 t0t0 time, t f t M-1 0 M -1 t0t0 67% of area between t 0 and t M-1

15. Some calculations If f(t) = c -1 exp( -t/c ) What value of c puts A 1 fraction of area between 0 and t 1 ? A =  0 t 1 c -1 exp( -t/c ) dt = exp(-t/c)| 0 t 1 = 1- exp(-t 1 /c) Note A(t=  )=1 So A 1 = 1- exp(-t 1 /c) (1-A 1 ) = exp(-t 1 /c) ln(1-A 1 ) = (-t 1 /c) c = -t 1 / ln(1-A 1 ) and t 1 = -c / ln(1-A 1 )

16. MatLab Code width = 14; % 67% width c = -width/log(1.0-0.67); % constant in exponential M = 6*round(width/dt); % truncate filter at 6 widths delay = -width/log(1.0-0.50); % delay at 50% of area f = exp( -t(1:M)/c ); amp = sum(f); % normalize to sum to unity f = f/amp;

17. Laguardia Airport Mean Temperature Blue: daily mean temperature data Red: 14 day Gaussian running average “Gaussian”“exponential”

18. first derivative filter f =  t [1, -1] T f = (  t) 2 [1, -2, 1] T second derivative filter But note delayed by  t But note delayed by ½  t M=2; f = dt*[1,-1]'; delay = 0.5*dt;

19. First-derivative note more-variable in winter

20. second-derivative note more-variable in winter

21. prediction error filter x = [x 1, x 2, x 3, x 4, … x N-1 ] T f = [-1, f 1, f 2, f 3, f 4, … f N-1 ] T Choose f such that f * x  0 f 5, f 4, f 3, f 2, f 1, -1 … x M-3, x M-2, x M-1, x M, x M+1, x M+2, x M+3, … x M = f 1 x M-1 + f 2 x M-2 + f 3 x M-3 … x M predicted from past values

22. MatLab Code M=10; % filter of length M, data of length N G = zeros(N+1,M);% solve by least-squares d = zeros(N+1,1);% implement condition f 0 =1 % as if its prior information for p = [1:M]% usual G matrix for filter G(p:N,p) = T(1:N-p+1); d(p)=0;% d vector is all zero end G(N+1,1)=1e6;% prior info, with epsilon=1e6 d(N+1)=-1e6; f = inv(G'*G)*G'*d;% least-squares solution y = conv(f,T);% try out fileter, y is prediction error

23. filter length M = 10 days filter, f

24. filter length M = 10 days f*xf*x x error = 6.4  10 5

25. filter length M = 100 days filter, f

26. filter length M = 10 days f*xf*x x error = 6.2  10 5

27. f * x is the unpredictable part of x

28. Let’s try it with the Neuse River Hydrograph Dataset Filter length M=100 What’s that? x f*xf*x

29. Close up of first year of data f*xf*x x Note that the prediction error, f * x, is spikier than the hydrograph data, x. I think that this means that some of the dynamics of the river flow is being captured by the filter, f, and that the unpredictable part is mostly the forcing, that is, precipitation