1. ASSESSING CREATIVE THINKING THROUGH PORTFOLIOS October 30, 2014 MAGGIE KONICH ASSESSMENT SPECIALIST

2. BIG PICTURE Not much data in this presentation! The process of assessment and how the OIRAE team can help you with long term development of evaluation.

3. SUPPLEMENTAL INSTRUCTION (SI) Supplemental Instruction (SI) began in Fall 2011 Course-specific, peer-led program that promotes: collaborative learning critical thinking transferable study skills Targets large courses with high DFW rates 1,533 unique attendees in Spring 2014 Fall 2014: 20 different courses and 30 student leaders

4. FRAMEWORK SI LEARNING OUTCOMES As a result of participating in SI, student leaders will learn: To develop creative thinking skills as they design lesson plans To enhance critical thinking skills as they set learning objectives To develop professionally by improving their oral communication skills To build their leadership skills as they facilitate group participation that encourages teamwork As a result of participating in SI, student leaders will learn: To develop creative thinking skills as they design lesson plans To enhance critical thinking skills as they set learning objectives To develop professionally by improving their oral communication skills To build their leadership skills as they facilitate group participation that encourages teamwork

5. FRAMEWORK SI LEARNING OUTCOMES creative thinking critical thinking oral communication leadership teamwork

6. SI Leaders for Spring 2014 SI Leaders Spring 2014

7. PORTFOLIO MATERIALS “5 Pro-Tips for Session Management” Five randomly selected lesson plans Observations “5 Top Activities” Two final reflections Portfolios allowed SI staff to balance structure and individuality, and were robust enough to capture what they wanted.

8. THE RUBRIC

9. Purpose: To capture the classroom management techniques and tricks that individual leaders developed throughout the semester Linked to: Innovative Thinking

10. 2 SESSION PLANS Purpose: To capture the leaders’ ability to form a cohesive plan and reflect on the benefits or drawbacks of that plan Linked to: Taking Risks; Solving Problems; Connecting, Synthesizing, and Transforming

11. 5 BEST ACTIVITIES Purpose: To capture how the leaders perform in real situations, and whether they are able to execute a cohesive session Linked to: Connecting, Synthesizing, Transforming

12. 2 FINAL REFLECTIO NS Purpose: To allow leaders to reflect on how they had improved, and what role risks and mistakes played in their development. Linked to: Taking Risks

13. THE RUBRIC

15. IMPLEMENTATION RESULTS THE GOOD Most leaders submitted very thoughtful and complete reflections and activities, which provided robust information The Portfolio is being used for other purposes in the program, such as staff training and knowledge archiving Both evaluators had: Final scores within the same designation for all SI leaders Category level scores within one point for all SI leaders

16. IMPLEMENTATION RESULTS NORMING CaseScore 1Score 2Designation 11517Outstanding 21616*Outstanding 315 Outstanding 411 Passing 599Failing 61214Passing 716 Outstanding * Indicates that each subscore between the two evaluators was identical Sample represents 30% of total portfolios Failing the assessment used as a development tool (not punitive)

17. IMPLEMENTATION RESULTS THE BAD Some leaders didn’t complete all worksheets Some leaders didn’t submit correct materials Quality of the worksheets would sometimes be poor Assembling portfolios took forever! Some wording could be confusing in the rubric

18. LESSONS LEARNED Be as specific in your directions as possible Let leaders know you intend to show the portfolio to others for more complete materials (like faculty) Add headers and footers so you can identify which leader submitted what paper Establish in advance how terms like “original, novel, and unique” will be determined Put the rubric in order of how the materials are organized (Tips  Reflections)

19. LONG STORY SHORT Step-by-Step Process: 1.Set Learning Outcomes 2.Plan Materials 3.Decide on a Process 4.Implement 5.Revise Benefits of a Portfolio: 1.Captures creative thinking by organizing the original works of the leader 2.Materials can also be used for various other uses, such as staff development and training Try This If: 1.You need to assess thorny, intangible elements. 2.You don’t have a lot of time to observe the population in person. 3.You have time to review each case. 4.You want a versatile tool that can be used for a variety of things

20. THANK YOU! Maggie Konich mkonich@purdue.edu For more about SI: https://www.purdue.edu/studentsuccess/academic/si/index.html