1. Assessing Computational Thinking
Eric Snow
Center for Technology in Learning
SRI International
Sangeeta Bhatnagar
Evergreen Valley High School

2. Overview What is computational thinking (CT)?
Key CS/CT assessment efforts to date.
Current CT Assessment Work at SRI International:
Planning for the Assessment of Computational Thinking (PACT 1)
ECS-Standards Alignment
Principled Assessment of Computational Thinking (PACT 2)

3. What is Computational Thinking?
Popularized by Jeanette Wing in ACM Viewpoints Article (2006)
“Universally applicable attitude and skill set everyone, not just computer scientists, would be eager to learn and use.”
Characteristics of CT:
Conceptualizing, not (just) programming
A way that humans, not computers, think
Ideas, not (just) artifacts
Draws on existing conceptions of mathematical, engineering (systems), and inquiry-based thinking
Not just for computer science/scientists

4. Key CS/CT Assessment Efforts to Date
CS Principles Computational Thinking Practices (College Board, 2011)
Connecting computing
Developing computational artifacts
Abstracting
Analyzing problems and artifacts
Communicating
Working effectively in teams
Aligned with new CS AP principles course
CS Concept Inventory (Herman, Loui, & Zilles, 2010)
FCS1 Assessment (Elliott Tew, 2010)
SUNY CS1-2 Exam (Decker, 2007)
FCS1 - Measures a set of foundational CS1 concepts that are common across a wide variety of current pedagogical approaches and paradigms

5. Planning for the Assessment of Computational Thinking (PACT 1)
NSF-CE21 Planning Grant (9/2011 – 6/2012)
Principal Investigators: Eric Snow, Marie Bienkowski
Project Goals
Advance field of assessment in computer science and computational thinking.
Leverage interdisciplinary expertise to begin creating an assessment framework and design templates for computational thinking (CT) aligned with the Exploring Computer Science curriculum.

6. Planning for the Assessment of Computational Thinking (PACT 1)
Project Plan
December 2011
Core expert group meets to begin creating an assessment framework and design templates for computational thinking (CT) aligned with the Exploring Computer Science curriculum
January 2012
Debrief core expert group meeting and circulate draft framework and design templates for internal review. Develop project dissemination web site. Begin planning for next stage of CT assessment design and development work
February – September 2012
Release project dissemination website, draft standards alignment document, invite CE21 community feedback on assessment framework, design templates and standards alignment work. Plan for next stage of CT assessment design and development work

8. Focal KSAs form the core of design templates that specify what we want to measure, the behaviors that will count as evidence of what we want to measure, and the types of tasks we can use to elicit the desired behaviors.

9. Planning for the Assessment of Computational Thinking (PACT 1)
ECS - CS/CT Standards Alignment
Objectives
Gain a better understanding of how ECS CT practices and learning objectives were related to relevant national and state standards (e.g., CSTA, Common Core).
Provide value to ECS team, especially teachers and others who had to convince stakeholders that ECS helps students learn valuable knowledge and skills
Worked with local ECS teacher to develop draft document that shows the alignment between ECS Unit learning objectives and CT practices, and CSTA, ISTE, Common Core, Next Generation Science/Engineering, and CA and IL state learning and CTE standards.
Currently under review/revision

10. ECS - CS/CT Standards Alignment

11. Planning for the Assessment of Computational Thinking (PACT 1)
Web Site URL:
Please note:
The site is still under construction.
PACT materials will continue to be posted over coming months. Please me if you would like to get an update on when materials are posted.
Feel free to send comments and feedback.

12. Principled Assessment of Computational Thinking (PACT 2)
NSF-CE21 Special Project (9/2012 – 8/2014)
Principal Investigators: Eric Snow, Marie Bienkowski
*proposal currently “Recommended” for funding
Project Goals
Advance field of assessment in computer science and computational thinking.
Leverage materials and partnerships developed under PACT 1 to design, develop and validate assessments of computational thinking for the ECS curriculum and, eventually for the larger CT community.

13. Principled Assessment of Computational Thinking (PACT 2)
Year 1
Extend and finalize domain analysis/modeling started in PACT 1; focus on ECS units 1-4, possibly unit 5 (“big data”).
Develop pilot assessment tasks for unit and summative assessments
Expert panel review of pilot tasks
Pilot summative assessment (include subset of unit tasks), including think-aloud interviews
Data analysis and assessment task revisions
Advisory panel

14. Principled Assessment of Computational Thinking (PACT 2)
Year 2
Expert panel review of summative and unit assessment tasks
Field test summative assessment, pilot unit assessments, including think-aloud interviews
Data analysis and final assessment task revisions
Advisory panel
Methodological approach focused on assessment design/development, and on collecting validity evidence based on test content, internal structure, response processes, and relations to other variables.

15. Computational Thinking Resources
National Research Council. (2004). Computer science: Reflections on the field. Washington, DC: The National Academies Press.
National Research Council Committee on the Workshops on Computational Thinking. (2010). Report of a workshop on the scope and nature of computational thinking. Washington, DC: National Academy Press.
National Research Council Committee on the Workshops on Computational Thinking. (2010). Report of a workshop on the pedagogical aspects of computational thinking. Washington, DC: National Academy Press.
CS Standards from International Society for Technology in Education (ISTE) and CS Teacher’s Association (CSTA) (2011)

16. THANK YOU!