1. Numerical Analysis 1 EE, NCKU Tien-Hao Chang (Darby Chang)

2. Correlation coefficient 2 Two continuous variables

3. Correlation coefficient (CC) What we need is a single summary number that answers the following questions: –does a relationship exist? –if so, is it a positive or a negative relationship? and –is it a strong or a weak relationship? Correlation coefficient, a single summary number that gives you a good idea about how closely one variable is related to another variable 3

4. Correlation coefficient Two-way scatter plot 4

5. 5 The mortality rate tends to decrease as the percentage of children immunized increase

6. Correlation Coefficient Pearsons correlation coefficient 6

7. 7

8. 8

9. Correlation Coefficient Correlation coefficient is not a percent 9

10. Correlation Coefficient Coefficient of determination 10

11. 11

12. Statistical test 12

13. Correlation coefficient Statistical inference 13

14. 14

15. Correlation coefficient Limitations It quantifies only the strength of the linear relationship between two variables Care must be taken when the data contain any outliers, or pairs of observations that lie considerably outside the range of the other data points A high correlation between two variables does not imply a cause-and-effect relationship 15

16. 16 http://upload.wikimedia.org/wikipedia/commons/thumb/e/ec/Anscombe%27s_quartet_3.svg/2000px-Anscombe%27s_quartet_3.svg.png

17. Correlation coefficient Spearmans rank CC 17

18. Any Questions? 18 About correlation coefficient

19. Statistical inference Basic tests –tests about proportions –tests about one mean –tests of the equality of two means –tests for variances –references http://zoro.ee.ncku.edu.tw/mlb2009/res/14-ch5.pdf (pp. 27-33)http://zoro.ee.ncku.edu.tw/mlb2009/res/14-ch5.pdf http://www.math.isu.edu.tw/finance/course/sta/ch8.ppt http://www.tnb.org.tw/Image/ttest.ppt http://www.mis.ncyu.edu.tw/course/download/cftai/Chapter%206.%20Continuous%20Probability%20 Distribution.PPThttp://www.mis.ncyu.edu.tw/course/download/cftai/Chapter%206.%20Continuous%20Probability%20 Distribution.PPT More advanced tests –ANOVA (analysis of variance) –goodness of fit (Wilcoxon test, Kolmogorov-Smirnov test, …) 19

20. Multivariate analysis Statistics –ANOVA –Multiple linear regression http://www.sjsu.edu/faculty/gerstman/biostat-text/Gerstman_PP15.ppt http://www.stat.nuk.edu.tw/Ray-Bing/regression/regression/Chapter3.ppt –PCA (principle component analysis) –ICA (independent component analysis) –LDA (linear discriminant analysis) So far, all techniques belong to statistics. You could find them in most statistical software, such as MATLAB, R (http://www.r-project.org/), SPSS…http://www.r-project.org/ Machine learning –Naïve Bayes (http://zoro.ee.ncku.edu.tw/mlb2009/res/11-ch4.pdf pp. 13-27)http://zoro.ee.ncku.edu.tw/mlb2009/res/11-ch4.pdf –LIBSVM (http://www.csie.ntu.edu.tw/~cjlin/libsvm/)http://www.csie.ntu.edu.tw/~cjlin/libsvm/ –RVKDE (http://mbi.ee.ncku.edu.tw/wiki/doku.php?id=rvkde)http://mbi.ee.ncku.edu.tw/wiki/doku.php?id=rvkde 20

21. 21 Lets see an Excel tutorial

22. 22 Lets see the data

23. 23 Points to a good final project

24. Raise some interesting issues –from observations –you have at least two trap issues (next slide) Design good analyses –make sure that your analyses fit your issues –do the results concur with your speculations? –design further analyses 24

25. Predict masked disease codes There are some masked diseases codes –for example, disease #14 has no acode, class and name First, predict the masked disease names Second, some masked diseases whose names are not in the file (namely, novel diseases). Try to identify them, and, if possible, to figure out what disease they are 25 acodeclass:name

26. The final project includes Presentation –slides (.ppt) and how you present them –convincing for me and your classmates –reasonably evaluate other works (voting others works if we have time) Project –scripts (executable) –results (.txt,.xls, …) –a step-by-step README of how you get the results from cd.dat (.txt,.doc, …) Report –a more detailed document of your slide (.doc) –the duty of each group member –anything worthy extra credit 26

27. Final grade Email all the materials to darby@mail.ncku.edu.tw before 2011/6/20 23:59darby@mail.ncku.edu.tw The raw grade will be available as soon as the final project of your group is received Ask me (darby@mail.ncku.edu.tw) about your grade with your NCKU email accountdarby@mail.ncku.edu.tw The final (adjusted) grades must wait all groups (2011/6/21, I hope) You have about one week to double-check the grade, and the final grades will be submitted around 2011/6/27 27