1. Update on ABET Assessment Work
Dr. Wasfi G. Al-Khatib
Information and Computer Science Department

2. Emerging Information Regarding ABET Assessment
Two major workshops related to ABET accreditation were given in the past two months.
“Major findings” out of these workshops are:
The assessment process must be useful in assessing the program outcomes, yet simple enough to be carried out on a regular basis
ABET DOES NOT require
Course-level learning outcome evaluation
although KFUPM as a school may do (see PAC guidelines).
Mapping each course-learning outcome to a program outcome

3. Current Direct Measure
The direct measure of using course learning outcomes, as has been advocated by the ABET steering committee earlier, raises the following concerns:
The use of course grades is highly discouraged by ABET for program outcome evaluation purposes (guess why?)
The system is complex and may not sustain periodic evaluation, which is essential for this and subsequent ABET visits (with all those weighted averages!!!!)
Not enough to substantiate the claims that students have fulfilled program outcomes based on individual course evaluation
Q: What is available for the evaluators from student work when they come to visit us to verify our achievement claims?
A: Only best and worst samples.

4. What to do??!!
Introduce and use “Rubrics” in addition to the current direct measure
A rubric is a descriptive rating scale with several different observable levels of performance possible for each performance criteria being assessed.
Each performance level is described and assigned a numeric score
For example, 4 = exemplary, 3 = good, 2 = marginal, 1 = unacceptable.
Why 4 not 5? (Make it simpler and reduce variations in evaluation)
Each performance level describes the level of cognition or skill that the outcome requires in order to be classified into this performance level.
It is a program outcome measure, but not a grade!!!!
Reporting the percent of students who score at each of the levels (or as an average, for example) provides useful data that are linked directly to the anticipated program outcome and focuses the evaluation in order to look for strategies for improvement, if needed.

5. Rubrics for Computer Science
Senior Project Option:
Senior Project:
Final Report
Final Presentation
Summer Training
Maybe the database course
Coop Option
Coop Course
Employer Input

6. Rubric Example from Computer Science (Early Draft)
Program
Outcome
4 points
3 points
2 points
1 point
Score
Modeling
Overall architecture present and well defined
The design of the system, broken into modules, is present and consistent
Details of each module are present and seem accurate
The design of the system, broken into modules, is present and mostly consistent
Details of 50% of modules are not present but seem accurate
The design of the system, broken into modules, is present but not complete or has inconsistencies.
Details of modules seem not to be accurate nor complete
The overall architecture may be present but the design of the system is either missing or lacking major components with little or no details

7. Rubric Example from Computer Science (Early Draft)
Program
Outcome
4 points
3 points
2 points
1 point
Score
Written Communication
Report is well organized
Report is grammatically sound
Report is spell-checked
Report is mostly grammatically sound
Some words are misspelled
Report contains major grammatical mistakes and/or some words are misspelled
Report is not well organized

8. Rubric Example from Computer Science (Early Draft)
Program
Outcome
4 points
3 points
2 points
1 point
Score
Oral Communication
Clear
Well-formed sentences
Confident
Maintains proper eye contact
Most sentences are well-formed
Not as clear
Not more than 50% of sentences are ill-formed
Not clear and/or many sentences are ill-formed.

9. Beyond Rubrics-attached Courses
Employer evaluation, in form of a questionnaire represent an excellent direct measure for “not-that-mappable” program outcomes to student work. For example
Ability to design a system/component to meet desired needs
Oral Communication
Team work and Professional/Ethical Responsibility
Contemporary issues
This can apply to COOP/Summer Training/Senior Project
This means that we need to change the current employer/senior project evaluation forms

10. Beyond Rubrics-attached Courses
Current Summer Training/Coop Form

11. Conclusions The department has to come up with
a set of courses to do a program evaluation by a group of faculty members.
Senior project, COOP, Summer Training and [may be] at most two other junior/senior-level courses
This will remove reliance on “grades”
a set of rubrics for each "student work".
This will canonize the program outcomes SCALE that is suitable for each department
This will reduce the variation in evaluation of different faculties
Remember that the using student reports and employer questionnaires can be displayed for each student for the team to look at and evaluate upon their visit
Whether you choose to do all senior projects or take a random sample
This needs to be done every semester.
A standing departmental committee has been established in each department to carry out the assessment every semester
We have to keep the current direct measure
We will evaluate it vs. rubrics and possibly stick to the most appropriate direct measure for future assessments

12. Thank You