1. Neural Network Design and Application
Fall 2012
CPTS 434 & 534, Tuesday and Thursday, Noon-1:15pm
Instructor: John Miller PhD
Associate Professor of Computer Science
Office: West 134E WSU Tri-Cities
Class web page can be found at
No Required Text
Suggested texts:
Building Neural Networks by David M. Skapura,
Introduction to Machine Learning, 2nd ed by Ethem Alpaydin
Neural Networks and Machine Leaning, 3rd ed by Simon Haykin

2. Graduate Project Reports:
Nuts and Bolts
Grades:
Tests and Assignments have equal weight
Graduate credit: project approved by instructor
Tests:
given in class with open books, lecture notes, and computers
Assignments:
Unless prior arrangements are made, assignments will not be accepted more than one week after their due date
Graduate Project Reports:
Topic approved by instructor
3 – 5 pages double spaced
Due last class period before dead week

3. More nuts and bolts
Accommodations for Disabled Students: 
Reasonable accommodations are available for students who have a documented disability. If you have a documented disability, even temporary, make an appointment as soon as possible with the Disability Services Coordinator,
Cherish Tijerina, ,
You will need to provide your instructor with the appropriate classroom accommodation form. The forms should be completed and submitted during the first week of class. Late notification may delay your accommodations. All accommodations for disabilities must be approved through Disability Services. Classroom accommodation forms are available through the Disability Services Office.

4. More nuts and bolts
Academic Integrity:
As stated in the WSU Tri-Cities Student Handbook," any member of the University community who witnesses an apparent act of academic dishonesty shall report the act either to the instructor responsible for the course or activity or to the Office of Student Affairs."
The Handbook defines academic dishonesty to include "cheating, falsification, fabrication, multiple submission [e.g., submitting the same or slightly revised paper or oral report to different courses as a new piece of work], plagiarism, abuse of academic material, complicity, or misconduct in research."
Infractions will be addressed according to procedures specified in the Handbook.

5. More nuts and bolts
Safety:
Should there be a need to evacuate the building (e.g., fire alarm or some other critical event), students should meet the instructor at the Cougar statue directly outside of the West building. A more comprehensive explanation of the campus safety plan is available at
The university emergency management plan is available at
Further, an alert system is available. You can sign up for emergency alerts (see through the zzusis site (

6. Student Concerns.
If you have any student concerns, you can contact Carol Wilkerson
the Director of Student Affairs in West 269F, (509) , or
If you have any concerns about this class, you should contact your
instructor first, if possible.
Attendance Policy.
Absences should be avoided. Students should contact an instructor if an
absence from class is unavoidable.
Students are encouraged to read Section 73 (Absences) of the
Washington State University Academic Regulations, which is found in the
WSU Tri-Cities Student Handbook.

7. Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011
new trends in the application of
machine learning

8. Rise and fall of supervised machine learning techniques, Jensen and Bateman, Bioinformatics 2011
Availability of sophisticated machine-learning
software packages, like WEKA, facilitates the
application of multiple methods to the same
problem

9. Objectives of the class:
Lean to apply artificial neural networks to classification
and regression problems
2. To understand artificial neural networks as a
non-parametric statistical method of data mining
3. To compare artificial neural networks to other supervised
machine-learning techniques

10. Tentative Schedule Tu Aug 21 Discussion of class syllabus
Th Aug 23 Introduction to supervised machine learning
Tu Aug 28 Introduction to supervised machine learning
Th Aug 30 Introduction to Bayesian statistics
Tu Sep 4 Introduction to Bayesian statistics
Th Sep 6 Parametric methods
Tu Sep 11 Parametric methods
Th Sep 13 Multivariate Data
Tu Sep 18 Multivariate Data
Th Sep 20 Test #1
Tu Sep 25 Artificial Neural Networks
Th Sep Artificial Neural Networks
Tu Oct 2 Artificial Neural Networks
Th Oct 4 Artificial Neural Networks
Tu Oct 9 Artificial Neural Networks
Th Oct 11 Genetic Algorithm
Tu Oct 16 Genetic Algorithm
Th Oct 18 Radial Basis Functions
Tu Oct 23 Radial Basis Functions
Th Oct 25 Self-Organizing Maps
Tu Oct 30 Self-Organizing Maps
Th Nov 1 Test #2.
Tu Nov 6 Advanced network designs
Th Nov 8 Advanced network designs
Tu Nov 13 Advanced network designs
Th Nov 15 Thanksgiving break
Tu Nov 20 Thanksgiving break
Th Nov 22 Support Vector Machines
Tu Nov 27 Support Vector Machines
Th Nov 29 Support Vector Machines
Dec 3-7 Review
Dec Finals week Test #3

11. Example of a Report-Type homework assignment
Dataset: Golub et al, Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring, Science, 286 (1999)
Download and become familiar with Weka software.
Open the leukemia gene expression data in Weka.
KNN technique is under the “lazy” menu of classifiers.
Weka refers to KNN as “IBk” for “Instance-Based k”.
After opening IBk, click on the text next to IBk to get a parameter menu. Set “KNN” to 5 and keep the default value of other parameters.
Under “Test options” choose “Cross-validation” with “Folds” equal to 5.
Include the following in your report:
Objective and conclusions of the paper
Nature and Structure of the input data
Results (include the performance metrics in lecture “foundations 2”)
Do your calculations support the authors’ conclusions

12. Example of a Programming-Type homework assignment
Generate 100 in silico data sets of 2sin(1.5x)+N(0,1)
each with 50 random x-values between 0 and 5
Use 50 data sets for training and 50 data sets for validation
Use the training data sets for polynomial regression of orders 1 – 5
For each order calculate the following:
RMS error for training data sets
RMS error for validation data sets
Bias squared
Variance
Plot your result as error vs order
Interpret your findings in terms of the “bias – variance dilemma”

13. Example of a Math-Type homework assignment
Derive the result
for Bayesian discriminant points in the 2-class problem with Gaussian class
likelihoods. Assume the mean and variance of C1 are 3 and 1, respectively.
Assume the mean and variance of C2 are 2 and 0.3, respectively.
For a sample size of 100, compare Bayesian discriminant points calculated
from maximum likelihood estimators with those derived from the true means
and variances.