A Qualitative Study of Animation Programming in the Wild Aniket Dahotre, Yan Zhang, Christopher Scaffidi ESEM 2010

2 Roles for animation programming A vehicle for getting kids excited about programming An environment for teaching programming skills A medium for communicating and entertaining A platform for research aimed at raising usability of programming tools Examples of animation programming tools: ○ Logo, KidSim, AgentSheets, Alice, Hands, Scratch Intro  Technical  Social  Remixing  Closing

3 Scratch as a particular programming tool Turing-complete language Events, loops, conditionals, sprites, sound… Drag-and-drop programming + Online community for sharing, trying, discussing & remixing animation projects Intro  Technical  Social  Remixing  Closing

4 Scratch online repository Intro  Technical  Social  Remixing  Closing

5 Statements and anecdotes from related work Scratch site is “the YouTube of interactive media ”. Users are “a new generation of creative, systematic thinkers comfortable using programming to express... ideas” Supports “ creative appropriation … the utilization of someone else’s creative work in the making of a new one” “The site’s collection of projects is wildly diverse, including video games, interactive newsletters, science simulations, virtual tours, birthday cards, animated dance contests, and interactive tutorials, all programmed in Scratch” One project obtained over 100 user comments. Several programmers formed collaborative partnerships. One particular Tetris game was remixed dozens of times. Intro  Technical  Social  Remixing  Closing

6 Questions for our study To what extent is Scratch succeeding as a basis for developing programming skills in the wild? ○ Technical programming skills ○ Social skills ○ Remixing/reuse skills Intro  Technical  Social  Remixing  Closing

7 Research methods Screen-scraped 100 randomly-selected animations ○ Including their code, usage statistics, and user comments Developed coding schemes for several research questions ○ One student examined subset of animations, proposed coding scheme ○ This student and another independently applied scheme to half ○ Negotiated modifications to coding scheme ○ Then checked each other’s work on the other half of the animations ○ < 10% disagreement; negotiated to resolve all disagreements ○ Coalesce codes to increase clarity of presentation as needed In a few cases, related work provided relevant coding schemes ○ Which we applied without modification A few research questions could be answered directly with quantitative (non-coded) data Intro  Technical  Social  Remixing  Closing

8 Functional roles of animations To teach skill or knowledge To entertainingly challenge To communicate fictional plot 56%: No clear functional role Intro  Technical  Social  Remixing  Closing

9 Use of programming constructs: repository vs. afterschool “Clubhouse” Intro  Technical  Social  Remixing  Closing

10 Apparent design patterns Controller methodWithLoop() Sprite onMessage() CollisionHandler onHit(type2) Sprite of type 1 CollisionHandler onHit(type1) onHit(type2) Sprite of type 2 AbstractHandler onHit(type1) onHit(type2) onHit(type3) Intro  Technical  Social  Remixing  Closing

11 Conclusion regarding technical skill development Relatively successful platform for development of technical programming skills ○ Comparable primitive use relative to Clubhouse users ○ Some demonstration of patterns (perhaps subconscious) ○ Comparable in complexity to spreadsheets and other programs created by end-user programmers (see paper for details) Intro  Technical  Social  Remixing  Closing

12 Kinds of user comments exchanged Approval + idea Approval, no suggestion Disapproval Other comments from users, other than project creator Comments from project creator Intro  Technical  Social  Remixing  Closing

13 Active collaboration on projects By “collaboration,” we mean ○ joint design or implementation ○ by a team of multiple people ○ consciously working on a common intellectual goal Reviewed user comments ○ E.g., to find comments like “I used your suggestion – thanks!” ○ E.g., or like “Thanks, my friend helped me with that.” ○ Also examined source code to look for any action on suggestions ○ Looked for comments revealing code edits by multiple people We found no such indications of any collaboration at all. Intro  Technical  Social  Remixing  Closing

14 Conclusion regarding social skill development Uneven success as platform for development of social programming skills ○ Half of projects received comments ○ Most comments were complimentary and offered suggestions ○ But unable to find evidence of actual collaboration Intro  Technical  Social  Remixing  Closing

15 Frequency of downloading and remixing Related work: 15% of projects were created by remixing But what fraction of projects are used to create remixes? Of our 100 projects… ○ 50% were downloaded at least once ○ 10% were remixed at least once ○ 5% were remixed at least once by other users Intro  Technical  Social  Remixing  Closing

16 Changes made to programs during remixing 20 of our 100 projects were created by remixing. Of these… Intro  Technical  Social  Remixing  Closing

17 Conclusion regarding remixing skill development Uneven success as platform for development of remixing skills ○ Most remixes simply involved multimedia tweaks ○ Few attempts at script modification during remixing ○ Nearly half of script modifications led to major bugs ○ Even the biggest modifications were still fairly small (see paper) ○ Frequency of remixing unimpressive vs other systems (see paper) Intro  Technical  Social  Remixing  Closing

18 Results and opportunities for future work Technical skill development: relatively successful ○ Perhaps still a need for helping animators to do higher-level design Social skill development: uneven success ○ Definitely a need for helping animators to collaborate Remixing skill development: uneven success ○ Definitely a need for helping animators to remix code Intro  Technical  Social  Remixing  Closing

19 Additional empirical questions for future work Do animators consciously understand design patterns? What usually happens after code downloads? Is some hard-to-detect collaboration somehow occurring? What kinds of interaction happen in the online forums (outside the context of particular projects)? How well do these programming skills transfer to other programming languages and tools? Intro  Technical  Social  Remixing  Closing

20 Thank You… For the opportunity to present. For your questions, thoughts, and constructive feedback. Intro  Technical  Social  Remixing  Closing

21 Frequency of interactions between users Intro  Technical  Social  Remixing  Closing

22 Size of repository animations by animations’ functional role