Neural Network Design and Application Fall 2012 CPTS 434 & 534, Tuesday and Thursday, Noon-1:15pm Instructor: John Miller PhD Associate Professor of Computer Science Office: West 134E WSU Tri-Cities jhmiller@tricity.wsu.edu Class web page can be found at http://www.tricity.wsu.edu/~jhmiller No Required Text Suggested texts: Building Neural Networks by David M. Skapura, Introduction to Machine Learning, 2nd ed by Ethem Alpaydin Neural Networks and Machine Leaning, 3rd ed by Simon Haykin

Graduate Project Reports: Nuts and Bolts Grades: Tests and Assignments have equal weight Graduate credit: project approved by instructor Tests: given in class with open books, lecture notes, and computers Assignments: Unless prior arrangements are made, assignments will not be accepted more than one week after their due date Graduate Project Reports: Topic approved by instructor 3 – 5 pages double spaced Due last class period before dead week

More nuts and bolts Accommodations for Disabled Students:  Reasonable accommodations are available for students who have a documented disability. If you have a documented disability, even temporary, make an appointment as soon as possible with the Disability Services Coordinator, Cherish Tijerina, 372-7352, ctijerina@tricity.wsu.edu You will need to provide your instructor with the appropriate classroom accommodation form. The forms should be completed and submitted during the first week of class. Late notification may delay your accommodations. All accommodations for disabilities must be approved through Disability Services. Classroom accommodation forms are available through the Disability Services Office.

More nuts and bolts Academic Integrity: As stated in the WSU Tri-Cities Student Handbook," any member of the University community who witnesses an apparent act of academic dishonesty shall report the act either to the instructor responsible for the course or activity or to the Office of Student Affairs." The Handbook defines academic dishonesty to include "cheating, falsification, fabrication, multiple submission [e.g., submitting the same or slightly revised paper or oral report to different courses as a new piece of work], plagiarism, abuse of academic material, complicity, or misconduct in research." Infractions will be addressed according to procedures specified in the Handbook.

More nuts and bolts Safety: Should there be a need to evacuate the building (e.g., fire alarm or some other critical event), students should meet the instructor at the Cougar statue directly outside of the West building. A more comprehensive explanation of the campus safety plan is available at http://www.tricity.wsu.edu/safetyplan/ The university emergency management plan is available at http://oem.wsu.edu/emergencies/ Further, an alert system is available. You can sign up for emergency alerts (see http://alert.wsu.edu) through the zzusis site (http://portal.wsu.edu/).

Student Concerns. If you have any student concerns, you can contact Carol Wilkerson the Director of Student Affairs in West 269F, (509) 372-7139, or carol.wilkerson@tricity.wsu.edu. If you have any concerns about this class, you should contact your instructor first, if possible. Attendance Policy. Absences should be avoided. Students should contact an instructor if an absence from class is unavoidable. Students are encouraged to read Section 73 (Absences) of the Washington State University Academic Regulations, which is found in the WSU Tri-Cities Student Handbook.

Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011 new trends in the application of machine learning

Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011 Availability of sophisticated machine-learning software packages, like WEKA, facilitates the application of multiple methods to the same problem

Objectives of the class: Lean to apply artificial neural networks to classification and regression problems 2. To understand artificial neural networks as a non-parametric statistical method of data mining 3. To compare artificial neural networks to other supervised machine-learning techniques

Tentative Schedule Tu Aug 21 Discussion of class syllabus Th Aug 23 Introduction to supervised machine learning Tu Aug 28 Introduction to supervised machine learning Th Aug 30 Introduction to Bayesian statistics Tu Sep 4 Introduction to Bayesian statistics Th Sep 6 Parametric methods Tu Sep 11 Parametric methods Th Sep 13 Multivariate Data Tu Sep 18 Multivariate Data Th Sep 20 Test #1 Tu Sep 25 Artificial Neural Networks Th Sep 27 Artificial Neural Networks Tu Oct 2 Artificial Neural Networks Th Oct 4 Artificial Neural Networks Tu Oct 9 Artificial Neural Networks Th Oct 11 Genetic Algorithm Tu Oct 16 Genetic Algorithm Th Oct 18 Radial Basis Functions Tu Oct 23 Radial Basis Functions Th Oct 25 Self-Organizing Maps Tu Oct 30 Self-Organizing Maps Th Nov 1 Test #2. Tu Nov 6 Advanced network designs Th Nov 8 Advanced network designs Tu Nov 13 Advanced network designs Th Nov 15 Thanksgiving break Tu Nov 20 Thanksgiving break Th Nov 22 Support Vector Machines Tu Nov 27 Support Vector Machines Th Nov 29 Support Vector Machines Dec 3-7 Review Dec 10-14 Finals week Test #3

Example of a Report-Type homework assignment Dataset: Golub et al, Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring, Science, 286 (1999) 531-537 Download and become familiar with Weka software. Open the leukemia gene expression data in Weka. KNN technique is under the “lazy” menu of classifiers. Weka refers to KNN as “IBk” for “Instance-Based k”. After opening IBk, click on the text next to IBk to get a parameter menu. Set “KNN” to 5 and keep the default value of other parameters. Under “Test options” choose “Cross-validation” with “Folds” equal to 5. Include the following in your report: Objective and conclusions of the paper Nature and Structure of the input data Results (include the performance metrics in lecture “foundations 2”) Do your calculations support the authors’ conclusions

Example of a Programming-Type homework assignment Generate 100 in silico data sets of 2sin(1.5x)+N(0,1) each with 50 random x-values between 0 and 5 Use 50 data sets for training and 50 data sets for validation Use the training data sets for polynomial regression of orders 1 – 5 For each order calculate the following: RMS error for training data sets RMS error for validation data sets Bias squared Variance Plot your result as error vs order Interpret your findings in terms of the “bias – variance dilemma”

Example of a Math-Type homework assignment Derive the result for Bayesian discriminant points in the 2-class problem with Gaussian class likelihoods. Assume the mean and variance of C1 are 3 and 1, respectively. Assume the mean and variance of C2 are 2 and 0.3, respectively. For a sample size of 100, compare Bayesian discriminant points calculated from maximum likelihood estimators with those derived from the true means and variances.