1. Supporting Creativity and User Interaction in CS 1 Homework Assignments Tammy VanDeGrift Associate Professor of Computer Science ACM SIGCSE 2015

2. Outline Background CS 1 Assignment Features Motivation Study Context, Participants, HW, Methods, Data Research Questions, Results and Discussion ACM SIGCSE 2015

3. CS 1 HW Options Lab exercises Programming to precise specifications Short-answer questions Tracing, debugging code CS 1 HW ACM SIGCSE 2015

4. Design Elements and Goals HW Design ElementGoal Specification includes required and open-ended parts Encourage creativity and customization Over half require user testsEncourage connection to the user Requires written documentation Encourage authority and ownership of process and product ACM SIGCSE 2015 Elmore, T. 2010. Generation iY: Our Last Chance to Save Their Future. Pet Gardener Publishing.

5. Study (Context) Institution: Course: 2 sections of CS 1 Spring 2012 Java Assumes no prior experience Participants: 44 students gave consent ACM SIGCSE 2015

6. Study (Research Questions) 1.Does allowing students to exercise creative freedom motivate students to learn and complete homework? 2.Does asking students to share their programs with non-programmers engage their learning? ACM SIGCSE 2015

7. Study (Homework Assignments) Homework AsstCreativityCS Concepts 1) Fortune TellerChoose fortuneVariables, I/O 2) MadlibChoose Madlib and where to place operations I/O, Strings 3) Greeting CardDesign graphics layoutI/O, graphics, selection 4) GolfDesign graphics layout; wind and speed Graphics, selection, iteration 5) Wheel of FortuneChoose puzzles; option spin wheel Random numbers, selection, iteration, arrays, methods 6) LibraryChoose classes to designUML, class design, methods, instance variables 7) Game PlayerChoose game player to model Classes, instance variables, methods, constructors 8) Image Transformation Choose image filtersArrays of objects, interfaces, loops 9) InheritanceChoose assignmentInheritance, class relationships 10) PalindromesChoose property for searchRecursion, file I/O, exceptions ACM SIGCSE 2015 For HW and starter code, visit: https://sites.up.edu/sigcse2015/

8. Study (Methods and Data) End-of-semester paper survey ▫Overall feedback about HW ▫User Tests ▫Open-ended aspects of HW Open-ended questions: used content analysis Aggregated numbers for quantitative questions ACM SIGCSE 2015

9. Results: Q1) Does allowing students to exercise creative freedom motivate students to learn and complete homework? ACM SIGCSE 2015 What did you like best about the HW?

10. Results: Q1) Does allowing students to exercise creative freedom motivate students to learn and complete homework? (continued) Creativity The homework assignments were challenging, yet, at the same time allowed us to be creative. Also, allowed us to implement the knowledge we, as students, have learned in class to create the program. Matched Material in Course I liked that they were well guided. The pseudocode really helped. I also liked that they go with what we are learning in class and also build upon all of the skills we've gained. The ability to be creative with the assignments is very nice as well. Skill Development I liked the part in that it enhanced your programming skills and show you what the field is like. The opportunity to apply the code and create programs with practical applications. ACM SIGCSE 2015

11. Results: Q1) Does allowing students to exercise creative freedom motivate students to learn and complete homework? (continued) ACM SIGCSE 2015 QuestionResult Did you enjoy the open-ended parts of the assignments? 41 / 44 said yes Did you complete additional optional enrichment? 27 / 44 said yes Did you write programs that were not part of the course? 15 / 44 said yes Favorite?HW 8, HW 4, HW 9 Least favorite?HW 6 (no coding) Completion rates505 / 510 assignments submitted

12. Results: Q1) Does allowing students to exercise creative freedom motivate students to learn and complete homework? (continued) ACM SIGCSE 2015 What factors motivated you to complete HW? 77% had motivation other than grades to complete HW

13. Results: Q1) Does allowing students to exercise creative freedom motivate students to learn and complete homework? (continued) What factors motivated you to complete HW? (sample answers) ACM SIGCSE 2015 grades, I want to get better at CS, fun, I could show my friends later, very satisfying feeling after completing the HW I wanted to get a chance to show what I have learned and put my skills to the test. Obviously, I did them for earning the grade as well. I found that I really enjoyed solving the problems set for me. I liked the time I spent programming. I really like coding and solving problems so I really enjoyed the HW

14. Results: Q2) Does asking students to share their programs with non-programmers engage their learning? ACM SIGCSE 2015 With whom did you share your programs? Average number of different people per student: 3.4 53% of users showed an interest in computer science

15. Results: Q2) Does asking students to share their programs with non-programmers engage their learning? (continued) ACM SIGCSE 2015 What concepts did you explain in the user tests? (free response)

16. Results: Q2) Does asking students to share their programs with non-programmers engage their learning? (continued) QuestionResult Did the required user test impact when you started HW assignments? Yes = 9, No = 1, Did not matter = 34 Did having a user run the program impact your effort? Yes = 17, No = 27 What surprised you when sharing programs with users? Catch bugs (6) Users tried invalid data (4) Interested in how code worked (4) Gave improvements (4) Enjoyed it (4) Helped with usability (2) User did not read directions (1) Easy to impress user (1) ACM SIGCSE 2015

17. Discussion Bias: 44 of 51 students gave consent, instructor did not look at surveys until after grades were submitted Creativity/Ownership: Over 25% stated in open-ended question that creativity was one of the features they liked about the HW Challenges: does not lend itself to automated grading, may be difficult for students to code to precise specification later in curriculum Benefits: students learn about ambiguity, students learn about defining requirements, students want to complete HW ACM SIGCSE 2015

18. Questions? We acknowledge the students who participated in this study. I want to thank Dr. Karen Eifler (School of Education, University of Portland) for fruitful discussions about this study. For the HW assignments, visit: https://sites.up.edu/sigcse2015/ ACM SIGCSE 2015