1. 數值方法 2008, Applied Mathematics NDHU 1  Nonlinear Recursive relations  Nonlinear recursive relations  Time series prediction  Hill-Valley classification  Lorenz series Neural Recursions

2. 數值方法 2008, Applied Mathematics NDHU 2 Nonlinear recursion Post-nonlinear combination of predecessors z[t]=tanh(a 1 z[t-1]+ a 2 z[t-2]+…+ a  z[t-  ])+ e[t], t= ,…,N

3. 數值方法 2008, Applied Mathematics NDHU 3 Post-nonlinear Projection tanh y

4. 數值方法 2008, Applied Mathematics NDHU 4 Data generation z[t]=tanh(a 1 z[t-1]+ a 2 z[t-2]+…+ a  z[t-  ])+ e[t], t= ,…,N tanh

5. 數值方法 2008, Applied Mathematics NDHU 5 Nonlinear Recursion LM learning

6. 數值方法 2008, Applied Mathematics NDHU 6 Prediction Use initial  -1 instances to generate the full time series (red) based on estimated post-nonlinear filter Post-nonlinear filter

7. 數值方法 2008, Applied Mathematics NDHU 7 UCI Machine Learning Repository Nonlinear recursions: hill-valley Classify hill and valley

8. 數值方法 2008, Applied Mathematics NDHU 8 Noise-free Hill Valley

9. 數值方法 2008, Applied Mathematics NDHU 9 Noise-free Hill Valley

10. 數值方法 2008, Applied Mathematics NDHU 10 Hill-Valley with noise

11. 數值方法 2008, Applied Mathematics NDHU 11 Hill-Valley with noise

12. 數值方法 2008, Applied Mathematics NDHU 12 Hill-Valley with noise

13. 數值方法 2008, Applied Mathematics NDHU 13 Nonlinear recursion LM_RBF: z[t]=F(z[t-1], z[t-2], …,z[t-  ])+ e[t], t= ,…,N LM_RBF 1

14. 數值方法 2008, Applied Mathematics NDHU 14 LM_RBF 1 LM_RBF 2

15. 數值方法 2008, Applied Mathematics NDHU 15 Nonlinear Recursion LM learning MLPotts 1 (H)

16. 數值方法 2008, Applied Mathematics NDHU 16 Nonlinear Recursion LM learning MLPotts 2 (V)

17. 數值方法 2008, Applied Mathematics NDHU 17 Approximating error LM_RBF 1 (H) Green: Original Hill Blue: Approximated Hill Red: Approximating error

18. 數值方法 2008, Applied Mathematics NDHU 18 Approximating error LM_RBF 2 (V) Green: Original Hill Blue: Approximated Hill Red: Approximating error

19. 數值方法 2008, Applied Mathematics NDHU 19 Separable 2D diagram of Hill-Valley Noise-free data set 1 LM_RBF 2 (V) LM_RBF 1 (H) Mean Approximating Errors of Two Models

20. 數值方法 2008, Applied Mathematics NDHU 20 Separable 2D diagram of Hill-Valley Noise-free data set 2 Mean Approximating Errors of Two Models LM_RBF 2 (V) LM_RBF 1 (H)

21. 數值方法 2008, Applied Mathematics NDHU 21 Separable 2D diagram of Hill-Valley Dataset 1 (Noise) Mean Approximating Errors of Two Models LM_RBF 2 (V) LM_RBF 1 (H)

22. 數值方法 2008, Applied Mathematics NDHU 22 Separable 2D diagram of Hill-Valley Dataset 2 (Noise) Mean Approximating Errors of Two Models LM_RBF 2 (V) LM_RBF 1 (H)

23. 數值方法 2008, Applied Mathematics NDHU 23 Chaos Time Series Eric's Home Page

24. 數值方法 2008, Applied Mathematics NDHU 24 Lorenz Attractor

25. 數值方法 2008, Applied Mathematics NDHU 25 Lorenz >> NN=20000; >> [x,y,z] = lorenz(NN); >> plot3(x,y,z,'.');

26. 數值方法 2008, Applied Mathematics NDHU 26

27. 數值方法 2008, Applied Mathematics NDHU 27 Nonlinear Recursion Neural nonlinear recursive relations

28. 數值方法 2008, Applied Mathematics NDHU 28 Prediction Use initial  -1 instances to generate the full time series (red) based on derived neural recursions

29. 數值方法 2008, Applied Mathematics NDHU 29 ρ=14, σ=10, β=8/3 ρ=13, σ=10, β=8/3 ρ=15, σ=10, β=8/3ρ=28, σ=10, β=8/3 Rayleigh number

30. 數值方法 2008, Applied Mathematics NDHU 30 Sunspot time series sunpot data load sunspot_train.mat; n=length(Y); plot(1712:1712+n-1,Y);

31. 數值方法 2008, Applied Mathematics NDHU 31 Sunspot time series sunpot data (testing) load sunspot_test_1.mat; n=length(Y); plot(1952:1952+n-1,Y);

32. 數值方法 2008, Applied Mathematics NDHU 32 S[1] S[2] S[3]. S[  -1] S[  ]. S[t-  ] S[t-  +1]. S[t-1] S[t] S[t+1]. x[t] y[t] x[  -1] y[  -1] Time-series 2 paired data

33. 數值方法 2008, Applied Mathematics NDHU 33 aMaM a2a2 a1a1 x Network tanh ……. 1 b2b2 b1b1 bMbM 1 r1r1 r2r2 rMrM r M+1 y

34. 數值方法 2008, Applied Mathematics NDHU 34 Paired data : tau=2 x=[]; len=3; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end figure; plot3(x(2,:),x(1,:),y,'.');hold on; y x

35. 數值方法 2008, Applied Mathematics NDHU 35 Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2:1712+2+length(y)-1,y,'r') MSE for training data 0.005029 ME for training data 0.052191K

36. 數值方法 2008, Applied Mathematics NDHU 36 Prediction

37. 數值方法 2008, Applied Mathematics NDHU 37 Sunspot Source codes

38. 數值方法 2008, Applied Mathematics NDHU 38 Paired data: tau=5 x=[]; len=5; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end

39. 數值方法 2008, Applied Mathematics NDHU 39 Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2:1712+2+length(y)-1,y,'r') ???

40. 數值方法 2008, Applied Mathematics NDHU 40 Example Repeat numerical experiments in slide #34 with Summarize your numerical results in a table

41. 數值方法 2008, Applied Mathematics NDHU 41 RBF y exp x..

42. 數值方法 2008, Applied Mathematics NDHU 42 addpath('..\..\..\Methods\FunAppr\LM_RBF'); % x : Nxd % y : 1xN N=200;d=2; x=rand(N,d)*2-1; y=tanh(-x(:,1).^2-x(:,2).^2); eval(['load data\sunspot_train.mat']); %([1 2 4 6 7 10],:) tx = X'; ty = Y; Net=LM_RBF(tx,ty); [yhat,D]=evaRBF(tx,Net); mean((ty-yhat).^2)/2

43. 數值方法 2008, Applied Mathematics NDHU 43 eval(['load data\sunspot_test_1.mat']); tex1 = X'; tey1 = Y; [yhat,D]=evaRBF(tex1,Net); mean((tey1-yhat).^2)/2 One_step_look_ahead prediction