Using Action Research to Guide Student Learning Experiences & Assessments Exploring the synergistic activities of curriculum design/delivery and assessment. Josephine C. Seddon, Student Learning Assessment Specialist Office of Institutional Research and Assessment Josephine C. Seddon, RPI-OIRA, 2015 Josephine C. Seddon, RPI-OIRA, 2015

Action Research Identify problem Use results to inform solution Plan action Collect and analyze data Reflect and evaluate Use results to inform solution Action research in teaching supports the identification of a problem and allows for exploration of solutions. Action research is cyclical in nature. Action research can be used by one or a group of colleagues exploring a teaching/learning/assessment related issue. Once the problem is identified, the next step is to “Plan Action” which includes identifying intended learning outcomes for the students based on your goals for the course or experience and with consideration for how the problem is to be solved. Choice of pedagogical approaches and assessment strategies will figure prominently in this stage. It is good to keep in mind too that pedagogical and assessment approaches may be tied together through embedded assessment opportunities or in the types of assessments that would best enable the learner to demonstrate achievement based on the type of experience in which they participated. Josephine C. Seddon, RPI-OIRA, 2015

Spotlight on Assessment Identify problem Plan action Collect and analyze data Reflect and evaluate Use results to inform solution Think about assessments and activities that will provide students with the opportunity to demonstrate that they are achieving or have achieved the learning outcomes. To consider…. Plans for formative and summative assessments that will also serve to inform teaching and learning as well as communicate level of achievement. *Types of assessments… (assignments, quizzes/tests/exams, presentations, projects, journals, performance tasks, portfolios, etc.) *How level of achievement will be identified… (test score, pass/fail, satisfactory/unsatisfactory, letter grade, ….) *How will the learner know what they need to improve upon? *How will you know you are being consistent in your grading? *What are your “look-fors”? Josephine C. Seddon, RPI-OIRA, 2015

Spotlight on Assessment Developing and Communicating Assessment Information “Look-fors” Formative and/or Summative Assessment Strategies A way to address the following is through the use of rubrics *What are your “look-fors”? *How level of achievement will be identified? *How will the learner know what they need to improve upon? *How will you know you are being consistent in your grading? Think also about the importance of being able to communicate assessment information with students or maybe even with colleagues. Highlight importance of role of formative assessment in teaching and learning and how rubrics can be very helpful in this respect Josephine C. Seddon, RPI-OIRA, 2015

Developing and Communicating Assessment Information Spotlight on Assessment Developing and Communicating Assessment Information Criteria Rubrics Level of Achievement Goal in creating a set of “look-fors” is to define criteria for grading and to determine what the different levels of achievement might “look like” in the context of the assessment piece. A rubric is helpful in that you can create a matrix that includes all of this information – and we will be looking at some examples of rubrics in a few minutes. Josephine C. Seddon, RPI-OIRA, 2015

Learning Outcomes & Assessment Use your set of course learning outcomes to help in the development of a rubric for your course. Consider how students will be provided with the opportunity to demonstrate achievement of the intended learning outcomes. Will you use in-class discussions, homework assignments, presentations, labs, quizzes, exams, reports, etc.? What will be the purpose of the assessment – formative/assessment? How will assessment instrument design support goals of formative assessment – i.e. communicating what the levels of achievement might look like and what is required in order to attain the next level? Josephine C. Seddon, RPI-OIRA, 2015

Developing and Communicating Assessment Information Spotlight on Assessment Developing and Communicating Assessment Information Criteria Rubrics Level of Achievement Creating a rubric using a set of criteria and descriptions for the various levels of achievement possible – or “look-fors” –will support communication of assessment results with students, make assessment transparent, and serve to inform both teaching and learning such that instructional strategies/content could be modified as needed to better address student/course needs and the student can work on challenge areas to further improve achievement and success in the course. Josephine C. Seddon, RPI-OIRA, 2015

Learning Outcomes & Assessment Looking again at the outcomes, choose one to focus on. For example: “Students will demonstrate the ability to apply biological concepts to issues of modern life.” To consider… How could the learning outcome be assessed? E.g. in-class discussions, quizzes, projects, (poster), presentations, research report, etc. How often could/should it be assessed? NB: formative assessment opportunities are very important if mastery learning is the goal. Students should have multiple opportunities to demonstrate achievement of the outcome and also have opportunity to revisit their learning and achievement with respect to the outcome after being given timely feedback on their progress toward achievement of the outcome. Also very important to consider in developing a rubric… What will you “look for” that will show that the student is achieving the outcome? For this example, you may need to look beyond the outcome slightly and think about the skills involved in being able to apply biological concepts to issues of modern life. You may also want to re-visit Bloom’s taxonomy. The verb used is “apply” so it could be limited to carrying out or using a procedure in a given situation, but you (or the professor in this case) might have actually meant to use a different word. This professor, for example, actually used classroom discussions on current scientific issues, such as, recent data reported in news media on global warming, human evolution, cancer research, stem cells, gene editing, and other topics for assessment of the (2nd) outcome. This actually implies that “critical thinking” was actually one of the skills necessary in students’ ability to apply biological concepts to issues of modern life. Josephine C. Seddon, RPI-OIRA, 2015

AAC&U Rubric – Critical Thinking Where to start in developing a rubric? AAC&U actually has a set of rubrics that you will probably want to become familiar with. We will begin with one of their rubrics to give us a running start. Please note that for each competency, the rubrics include: definition for the skill, potential categories for the skill, example levels of achievement for each category, and the “look-fors” within each. Q: Could any/all of these “look-fors” be used in assessment of the 2nd outcome for the biology course: “Students will demonstrate the ability to apply biological concepts to issues of modern life.” If not, we can adapt the category/achievement details to better reflect the course goals and intended outcomes. We can also modify the wording/design of assessment instrument so that students have an opportunity to demonstrate achievement of the outcome based on the criteria and indicators for level of achievement. Josephine C. Seddon, RPI-OIRA, 2015

Advantages/Challenges? “Rubrics can help instructors communicate their expectations to students and assess student work fairly and efficiently.  Rubrics can also provide students with informative feedback on their strengths and weaknesses, and prompt students to reflect on their own work.” Additionally, rubrics can provide a foundation for designing a self-assessment tool that could serve as formative assessment tool to support mastery learning. Josephine C. Seddon, RPI-OIRA, 2015

Steps in designing and assessing learning Evolution of a rubric… Steps in designing and assessing learning 1. Review the learning outcome(s). 2. Consider the skills/competencies necessary for achievement of the outcome. 3. Design learning opportunities that will support achievement of the intended outcomes. 4. Consider the purpose of the assessment for which you are creating a rubric. 4. Consider how the assessment will provide opportunity for students to demonstrate level of achievement for the intended learning outcome. 5. Design the rating scale, and write descriptions for each level of the rating scale 6. Consider how the learner will be provided with the opportunity to demonstrate achievement, and design assessment accordingly. 7. Publish and share your rubric. The evolution of a grading rubric… Review the learning outcome(s). Define the purpose of the assignment/assessment for which you are creating a rubric. Consider the skills/competencies involved and relevant categories.  Design the rating scale (1,2,3,4; emerging/novice/expert; etc).  Write descriptions for each level of the rating scale (“look-fors”). Consider how the learner will be provided with the opportunity to demonstrate achievement, and design assessment accordingly --- this is very important and factors in to multiple choice quizzes, project assignment design, discussion opportunities/prompts, etc. Publish and share your rubric AND…use the rubric in your feedback to/with students as a communication piece and for formative purposes (i.e. to inform both teaching and learning)   Josephine C. Seddon, RPI-OIRA, 2015

FOLLOW UP ACTIVITY… Review YOUR learning outcome(s) and choose one OR use the assessment piece that you brought with you and determine the learning outcome you believe it was intended to assess. Consider the skills/competencies necessary for achievement of that outcome and develop categories (definitions) that would help describe/define the skills/competencies involved. Design a rating scale, and write descriptions or “look-fors” for each level of the rating scale. Consider how the learner will be provided with the opportunity to demonstrate achievement on the assessment. What would you be sure to include in the design of the assessment? Consider follow-up opportunities for additional learning and/or assessment to support greater levels of achievement of the learning outcome(s). This can be done individually or partner/group. Working together in professional learning communities is often recommended. *Please consider sharing “likes”, “observations”, questions and challenges, etc. when working with a partner or in reflecting on your own rubric development. Be willing to ask yourself: “What did you notice?” Support is also always available from Josephine Seddon (carneg2@rpi.edu) in the Office of Institutional Research and Assessment. Please feel free to send a note or call 276-3884 for a follow-up meeting so we can collaborate together! ~Josie Josephine C. Seddon, RPI-OIRA, 2015