1. Neural Network Design and Application Fall 2015 CPTS 434 & 534 Instructor: John Miller Office: West 134E WSU Tri-Cities jhmiller@tricity.wsu.edu Class web page can be found at http://users.tricity.wsu.edu/~jhmiller Required Text: Learning from Data by Abu-Mostafa, Magdom-Ismail and Lin Suggested texts: Building Neural Networks by David M. Skapura, Introduction to Machine Learning, 2 nd ed by Ethem Alpaydin Neural Networks and Machine Leaning, 3 rd ed by Simon Haykin

2. Grades: Tests and Assignments have equal weight Tests: quizzes and final exam given in class with open books, lecture notes, and computers Assignments: Prior approval is required for late submission. Full credit on resubmissions until tested on subject matter. 50% credit thereafter. Graduate Project Reports: Topic approved by instructor 3 – 5 pages double spaced Due last class period before dead week Nuts and Bolts

3. IMPORTANT: Per new WSU policy effective August 24, I will ONLY be able to respond to emails sent from your WSU email address. I will NOT be able to respond to emails sent from your personal email address as of the first day of fall semester. Effective the 24th, the IT Department will switch the “preferred” email address in your myWSU to your WSU email address. More nuts and bolts

4. Accommodations for Disabled Students: Reasonable accommodations are available for students who have a documented disability. If you have a documented disability, even temporary, make an appointment as soon as possible with the Disability Services Coordinator, Cherish Tijerina, 372-7352, ctijerina@tricity.wsu.eductijerina@tricity.wsu.edu You will need to provide your instructor with the appropriate classroom accommodation form. The forms should be completed and submitted during the first week of class. Late notification may delay your accommodations. All accommodations for disabilities must be approved through Disability Services. Classroom accommodation forms are available through the Disability Services Office. More nuts and bolts

5. Academic Integrity: As stated in the WSU Tri-Cities Student Handbook," any member of the University community who witnesses an apparent act of academic dishonesty shall report the act either to the instructor responsible for the course or activity or to the Office of Student Affairs." The Handbook defines academic dishonesty to include "cheating, falsification, fabrication, multiple submission [e.g., submitting the same or slightly revised paper or oral report to different courses as a new piece of work], plagiarism, abuse of academic material, complicity, or misconduct in research." Infractions will be addressed according to procedures specified in the Handbook. More nuts and bolts

6. Safety: Should there be a need to evacuate the building (e.g., fire alarm or some other critical event), students should meet the instructor at the Cougar statue directly outside of the West building. A more comprehensive explanation of the campus safety plan is available at http://www.tricity.wsu.edu/safetyplan/ The university emergency management plan is available at http://oem.wsu.edu/emergencies/ Further, an alert system is available. You can sign up for emergency alerts (see http://alert.wsu.edu) through the zzusis site (http://portal.wsu.edu/). More nuts and bolts

7. Student Concerns. If you have any student concerns, you can contact Carol Wilkerson the Director of Student Affairs in West 269F, (509) 372-7139, or carol.wilkerson@tricity.wsu.edu.carol.wilkerson@tricity.wsu.edu If you have any concerns about this class, you should contact your instructor first, if possible. Attendance Policy. Absences should be avoided. Students should contact an instructor if an absence from class is unavoidable. Students are encouraged to read Section 73 (Absences) of the Washington State University Academic Regulations, which is found in the WSU Tri-Cities Student Handbook. More nuts and bolts

8. Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011 Predominance of ANN has diminished

9. Can be used as a black box Makes nonlinear modeling easy Magic due to biological basis Why was ANN so popular?

10. Applications of ANN by subject From “Neural Network Design” by Hagan, Demuth and Beale

11. Applications of ANN by subject

13. Objectives of the class: 1. To learn the general principals of data mining 2. Lean to apply artificial neural networks to classification and regression problems 3. To compare artificial neural networks to other supervised machine-learning techniques

14. Topics: Basic data mining fundamentals of machine learning linear models high-order polynomial models overfitting and regularization dimensionality reduction clustering ANN perceptron multi-layer perceptron feed-forward ANN radial basis function ANN Other techniques self organizing maps support vector machines

15. Example of a Report-Type homework assignment Dataset: Golub et al, Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring, Science, 286 (1999) 531-537 Download and become familiar with Weka software. Open the leukemia gene expression data in Weka. KNN technique is under the “lazy” menu of classifiers. Weka refers to KNN as “IBk” for “Instance-Based k”. After opening IBk, click on the text next to IBk to get a parameter menu. Set “KNN” to 5 and keep the default value of other parameters. Under “Test options” choose “Cross-validation” with “Folds” equal to 5. Include the following in your report: Objective and conclusions of the paper Nature and Structure of the input data Results (include the performance metrics) Do your calculations support the authors’ conclusions

16. Example of a Programming-Type homework assignment Generate 100 in silico data sets of 2sin(1.5x)+ N (0,1) each with 50 random x-values between 0 and 5 Use 50 data sets for training and 50 data sets for validation Use the training data sets for polynomial regression of orders 1 – 5 For each order calculate the following: RMS error for training data sets RMS error for validation data sets Bias squared Variance Plot your result as error vs order Interpret your findings in terms of the “bias – variance dilemma”

17. Derive the result for Bayesian discriminant points in the 2-class problem with Gaussian class likelihoods. Assume the mean and variance of C1 are 3 and 1, respectively. Assume the mean and variance of C2 are 2 and 0.3, respectively. For a sample size of 100, compare Bayesian discriminant points calculated from maximum likelihood estimators with those derived from the true means and variances. Example of a Math-Type homework assignment

19. Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011 Availability of sophisticated machine-learning software packages, like WEKA, facilitates the application of multiple methods to the same problem

20. Tentative Schedule Tu Aug 21Discussion of class syllabus Th Aug 23Introduction to supervised machine learning Tu Aug 28Introduction to supervised machine learning Th Aug 30Introduction to Bayesian statistics Tu Sep 4Introduction to Bayesian statistics Th Sep 6Parametric methods Tu Sep 11Parametric methods Th Sep 13Multivariate Data Tu Sep 18Multivariate Data Th Sep 20Test #1 Tu Sep 25Artificial Neural Networks Th Sep 27 Artificial Neural Networks Tu Oct 2Artificial Neural Networks Th Oct 4 Artificial Neural Networks Tu Oct 9Artificial Neural Networks Th Oct 11Genetic Algorithm Tu Oct 16Genetic Algorithm Th Oct 18Radial Basis Functions Tu Oct 23Radial Basis Functions Th Oct 25Self-Organizing Maps Tu Oct 30Self-Organizing Maps Th Nov 1Test #2. Tu Nov 6Advanced network designs Th Nov 8Advanced network designs Tu Nov 13Advanced network designs Th Nov 15Thanksgiving break Tu Nov 20Thanksgiving break Th Nov 22Support Vector Machines Tu Nov 27Support Vector Machines Th Nov 29Support Vector Machines Dec 3-7Review Dec 10-14Finals week Test #3