Assessing Computational Thinking Eric Snow Center for Technology in Learning SRI International Sangeeta Bhatnagar Evergreen Valley High School

Overview What is computational thinking (CT)? Key CS/CT assessment efforts to date. Current CT Assessment Work at SRI International: Planning for the Assessment of Computational Thinking (PACT 1) ECS-Standards Alignment Principled Assessment of Computational Thinking (PACT 2)

What is Computational Thinking? Popularized by Jeanette Wing in ACM Viewpoints Article (2006) “Universally applicable attitude and skill set everyone, not just computer scientists, would be eager to learn and use.” Characteristics of CT: Conceptualizing, not (just) programming A way that humans, not computers, think Ideas, not (just) artifacts Draws on existing conceptions of mathematical, engineering (systems), and inquiry-based thinking Not just for computer science/scientists

Key CS/CT Assessment Efforts to Date CS Principles Computational Thinking Practices (College Board, 2011) Connecting computing Developing computational artifacts Abstracting Analyzing problems and artifacts Communicating Working effectively in teams Aligned with new CS AP principles course CS Concept Inventory (Herman, Loui, & Zilles, 2010) FCS1 Assessment (Elliott Tew, 2010) SUNY CS1-2 Exam (Decker, 2007) FCS1 - Measures a set of foundational CS1 concepts that are common across a wide variety of current pedagogical approaches and paradigms

Planning for the Assessment of Computational Thinking (PACT 1) NSF-CE21 Planning Grant (9/2011 – 6/2012) Principal Investigators: Eric Snow, Marie Bienkowski Project Goals Advance field of assessment in computer science and computational thinking. Leverage interdisciplinary expertise to begin creating an assessment framework and design templates for computational thinking (CT) aligned with the Exploring Computer Science curriculum.

Planning for the Assessment of Computational Thinking (PACT 1) Project Plan December 2011 Core expert group meets to begin creating an assessment framework and design templates for computational thinking (CT) aligned with the Exploring Computer Science curriculum January 2012 Debrief core expert group meeting and circulate draft framework and design templates for internal review. Develop project dissemination web site. Begin planning for next stage of CT assessment design and development work February – September 2012 Release project dissemination website, draft standards alignment document, invite CE21 community feedback on assessment framework, design templates and standards alignment work. Plan for next stage of CT assessment design and development work

Focal KSAs form the core of design templates that specify what we want to measure, the behaviors that will count as evidence of what we want to measure, and the types of tasks we can use to elicit the desired behaviors.

Planning for the Assessment of Computational Thinking (PACT 1) ECS - CS/CT Standards Alignment Objectives Gain a better understanding of how ECS CT practices and learning objectives were related to relevant national and state standards (e.g., CSTA, Common Core). Provide value to ECS team, especially teachers and others who had to convince stakeholders that ECS helps students learn valuable knowledge and skills Worked with local ECS teacher to develop draft document that shows the alignment between ECS Unit learning objectives and CT practices, and CSTA, ISTE, Common Core, Next Generation Science/Engineering, and CA and IL state learning and CTE standards. Currently under review/revision

ECS - CS/CT Standards Alignment

Planning for the Assessment of Computational Thinking (PACT 1) Web Site URL: http://pact.sri.com Please note: The site is still under construction. PACT materials will continue to be posted over coming months. Please email me if you would like to get an update on when materials are posted. Feel free to send comments and feedback.

Principled Assessment of Computational Thinking (PACT 2) NSF-CE21 Special Project (9/2012 – 8/2014) Principal Investigators: Eric Snow, Marie Bienkowski *proposal currently “Recommended” for funding Project Goals Advance field of assessment in computer science and computational thinking. Leverage materials and partnerships developed under PACT 1 to design, develop and validate assessments of computational thinking for the ECS curriculum and, eventually for the larger CT community.

Principled Assessment of Computational Thinking (PACT 2) Year 1 Extend and finalize domain analysis/modeling started in PACT 1; focus on ECS units 1-4, possibly unit 5 (“big data”). Develop pilot assessment tasks for unit and summative assessments Expert panel review of pilot tasks Pilot summative assessment (include subset of unit tasks), including think-aloud interviews Data analysis and assessment task revisions Advisory panel

Principled Assessment of Computational Thinking (PACT 2) Year 2 Expert panel review of summative and unit assessment tasks Field test summative assessment, pilot unit assessments, including think-aloud interviews Data analysis and final assessment task revisions Advisory panel Methodological approach focused on assessment design/development, and on collecting validity evidence based on test content, internal structure, response processes, and relations to other variables.

Computational Thinking Resources National Research Council. (2004). Computer science: Reflections on the field. Washington, DC: The National Academies Press. National Research Council Committee on the Workshops on Computational Thinking. (2010). Report of a workshop on the scope and nature of computational thinking. Washington, DC: National Academy Press. National Research Council Committee on the Workshops on Computational Thinking. (2010). Report of a workshop on the pedagogical aspects of computational thinking. Washington, DC: National Academy Press. CS Standards from International Society for Technology in Education (ISTE) and CS Teacher’s Association (CSTA) (2011)

THANK YOU!