1. Reflective Assessment in Computing Instruction
Doug Peterson
Greater Essex County District School Board, Ontario CA
J. Philip East
University of Northern Iowa
Much of the time for many of us, assessment means grading student work so we can give them a grade for a course. We all know there can be more than that but often there is too little time to do more. We don’t plan to criticize anyone’s practice or to offer a miracle that gives you more time.
We do hope that something we say or do will spark a thought for some of you that may lead to some change in your thinking or practice that improves education in your case. The goal of this session is twofold: a) to illustrate some student assessment ideas and b) to encourage the practice of reflective assessment, both of students and of CS & IT instruction.

2. Overview Reflective Practice (what & why) Assessment Overview
Assessment Strategies/Tools/Examples
Discussion (reflection on session)
While we will have some general discussion the hope is that our examples and discussion will provide something that you can use in your teaching or that spurs a thought of something you might try. For those of you who like the general discussion of such topics, get in touch with us after the session or via --we like to talk ideas. Our other goal is to encourage reflection in teaching. A reasonable thing to start reflecting on in teaching is student assessment.
As you can see from the slide, we will begin with a short discussion of reflective practice and then some general ideas about assessment. Much of the session will examine a number of examples of student assessment in contexts that vary in level and subject matter or content. We will close by addressing both assessment and reflection.
Please feel free to offer discussion and questions at most any time. Good teachers encourage questioning and discussion and are prepared to determine when not to pursue it.
Questions before we get started?

3. Reflective Practice What? Why? How?
A starting place -- student assessment
One of the current buzz words in education is reflection or reflective practice. What is reflective practice? We are unaware of any official definition. It seems to us, however, that reflective practice includes thinking about the practice of teaching both before and after the teaching. Thinking before provides the opportunity to consider what teaching activity, assignments, etc. will be effective for student learning. Thinking after practice provides us with the opportunity to consider how well we performed the activity or how well the assignment worked, etc.
If we are considering reflective practice of assessment then we will be thinking (before the assessment) about what form of assessment will best accomplish our goal or (after the assessment) about how well the assessment was implemented and how well it worked for the goals we had in mind.
Does good teaching require reflective practice? We think not. However, “better” teaching probably does Michael Jordan probably started out being fairly good at playing basketball. Had he merely played a bunch of pick-up games he would likely have gotten even better. But, with some coaching, reflection about what/how/why to perform, practice in the performance, and reflection on the practice, he became an outstanding basketball player--certainly significantly better than he would have been without the reflective practice. Reflective practice of teaching is similar for those seeking to improve their teaching.
So what does reflective practice involve? Well, we think it mostly involves asking many questions both before and after the practice. Before you do a thing (while planning) ask about or consider your goal(s), why the particular approach should work, what alternatives exist, why they would work less well, etc. After you complete the thing ask if it seemed to work, if the reasoning about why it works still makes sense, alternatives or modifications that might be better, were the goals really appropriate, …
One should not expect to address the whole of their teaching activity when reflecting on their practice. Rather, it seems most useful to pick one or two things to focus on. In our experience, most teaching activities are so interrelated that a single focus will touch many aspects of teaching.

4. Purpose of Assessment A very “rich” question Content being assessed
Assessment versus/and instruction
A couple examples
HTML and Web page design & implementation
Programming
“What is the purpose of assessment?” is a very rich question. A lot can come of addressing the question. Assessment can be formative or summative. If summative, it might be designed primarily to yield a grade or to determine whether students learned as desired OR the teacher’s teaching (or both). Assessment can be formative--meant primarily to provide feedback to students on how to improve their performance or understanding. In this session we will mostly be addressing assessments relating to student learning or performance.
So a critical first/early step in reflective assessment is to clarify/explicate the general purpose of the assessment: summative?, formative?, both?, for the teacher?, for the student?.
You will also need to determine the “content” of the assessment. This seems relatively straight-forward, but may not be. For example, if we want students to learn spreadsheets they need to learn what spreadsheets can do, how to make them do desired tasks, and what tasks are needed to accomplish the overall desired result. On which of these should we assess the students?
Many now believe assessment should be addressed first when designing instruction. While the actual assessment will occur during the activity or after students have submitted some artifact, the parameters of the assessment were determined when the activity was designed. Thus, the learning goals will be better defined.
We have a couple examples that we’d like to use to illustrate some ideas about assessment and the close tie between instruction and assessment and the purpose of assessment. We will present two assessment tools that show different approaches to assessment. One tools was developed for and IT assignment and one for a programming assignment. However, either kind of tool can be used in either place.

5. Example 1 (Philip) http://www. cns. uni
Fluency class—Web page design & implementation using hand-coded HTML (see )
Use of the rubric
Influence of the rubric
Discussion (Doug, audience)
This first example is one in which the students are expected to learn and use a variety of HTML codes as they design and implement a Web page. Actually, the design is checked/assessed separately and mostly addresses skill at scanning a drawing and making it available on the Web and giving some relatively thought to what they want the sight to look like, i.e., actually designing it a bit rather than just throwing something together. The assignment description provided the students does not include this actual rubric but it does provide specifications and indicate the basis for assessment.
When I use the rubric I first check the basic requirements. Most recently I deducted points for each failure, with each HTML or submission feature counting 1 point, giving appropriate credit 2 or more points depending on how big a problem there was, and hand-craftedness counting a bunch of points (perhaps 10). Next, I checked the students self-assessment. If it existed and included some sort of rationale beyond “I tried” or “I spent a lot of time”. I awarded 1-4 points and went on to check additional quality indicators. For each, I consider whether there were problems or seemed okay but nothing of note or was quite good and provided “scores” from -2 to +2 in each area. Generally, students got -1, 0, or +1 points for each.
Over several offerings of the course I have been disappointed in the actually quality of student work. The design is rather marginal and students seems to do a few flashy things with pictures but or a similar thing over and over but not really consider the qualities I suggest are important. I give a similar kind of assignment concerning spreadsheet and issue examination projects with similar low-quality results. My reflection on why this is so have led me to believe it is a matter of my teaching and student motivation or capability at design or (with issue projects) analysis and argument. So, I have tried (with the HTML project) and will try with the spreadsheet and issue projects to encourage students to actually design what they want. We discuss Web design in class (and I show a better example) and I plan to provide some critique of design work and analysis of future projects. Thus, my attention to and reflection on assessment have actually caused a change in my instruction. I now (plan to) more directly address the design issues and provide students with feedback (formative assessment) on design that will, I believe, result in higher quality end-products.
Doug:

7. Example 2 (Doug) http://www. cns. uni
A general programming assignment rubric
Use of the rubric
Influence of the rubric
Discussion (Philip, audience) …
Philip:
This kind of rubric is probably more common than the kind I showed previously. I think it can be quite useful though the example we provide here is general to programming rather than specific to a particular assignment, which I think rubrics need to be. Certainly one could relatively easily produce a rubric of this type from the one that I used for the HTML project, perhaps something like:
Level 1 (specifications not met): is not hard-crafted or omits a substantial number of HTML features
Level 2 (specs nearly met): is hand-crafted, meets most HTML specs, and most submission specs
Level 3 (specs met, satisfactory result): is hand-crafted, includes all required HTML features, meets
submission specs and includes reasonable self-assessment
Level 4 (good work): in addition to level 3 satisfaction, design and implementation exceeds specifications
somewhat in many cases or exceeds them significantly in several cases
Level 5 (excellent work): in addition to level 3 satisfaction, design and implementation exceeds specs
significantly in many/most cases
Back to the original “programming” rubric, in addition to making it apply to a particular assignment, I would also restrict my focus on a particular assignment. For example, I would always address program correctness but would only examine one or two other aspects of program quality on a particular assignment. My reflection on feedback to students is that if we supply information on too many topics, students do not know which things to attend to. So while I might keep everything on the rubric (so students have a complete idea of program quality) I would just “grade” particular assignments on one or two things beyond correctness. Of course, really awful things not being focused on would be subject to feedback and perhaps value deductions.
My own elements of program quality are: correctness (works correctly and does correct things); solution design (clarity, efficiency, modularity); implementation/coding style (lack of code duplication, language feature use, etc.); code layout; documentation (code and comments); and user interface.

8. Questions/Comments/Discussion