1. © 2013 SRI International - Company Confidential and Proprietary Information Center for Technology in Learning SRI International NSF Showcase 2014 SIGCSE Conference Atlanta, GA Eric Snow & Marie Bienkowski Principal Investigators

2. © 2013 SRI International - Company Confidential and Proprietary Information How can we improve CS teaching, learning, and adoption through evidence- centered assessment?

3. © 2013 SRI International - Company Confidential and Proprietary Information PACT Project Goals I  Develop valid and reliable assessments of computational thinking practices  Aid in the adoption of high school computer science courses through assessments that stakeholders recognize as useful boundary objects

4. © 2013 SRI International - Company Confidential and Proprietary Information PACT Project Goals II  Create design patterns for computational thinking practices that can be used to develop new assessments as curriculum evolves  Create and field test assessments for Exploring Computer Science (ECS)

5. © 2013 SRI International - Company Confidential and Proprietary Information

7. Evidence-Centered Assessment Design (ECD)  What complex of knowledge, skills, or other attributes should be assessed?  What behaviors or performances should reveal those constructs?  What tasks or situations should elicit those behaviors?

8. © 2013 SRI International - Company Confidential and Proprietary Information  From Mislevy & Riconscente, 2006 Assessment Delivery Students interact with tasks, performances evaluated, feedback created. Four- process delivery architecture. Assessment Implementation Conceptual Assessment Framework Domain Modeling Domain Analysis What is important about this domain? What work and situations are central in this domain? What KRs are central to this domain? What is important about this domain? What work and situations are central in this domain? What KRs are central to this domain? How do we represent key aspects of the domain in terms of assessment argument. Conceptualization. Design structures: Student, evidence, and task models. Generativity. Manufacturing “nuts & bolts”: authoring tasks, automated scoring details, statistical models. Reusability.

9. © 2013 SRI International - Company Confidential and Proprietary Information Computational Thinking Practices Non- Cognitive Skills Inquiry Skills CS Concepts

10. © 2013 SRI International - Company Confidential and Proprietary Information Computational Thinking Practices Example CS Concepts Example Inquiry Skills Example Non-Cognitive Skills AlgorithmsEvaluate Communication ProgrammingExplore Teamwork/collaboration RecursionAnalyze Leadership AbstractionExplain Self-efficacy Debugging / TestingElaborate Persistence VariablesModel Organization Integration

11. © 2013 SRI International - Company Confidential and Proprietary Information PACT Project Accomplishments I  Aligned Exploring Computer Science lesson objectives to CSTA, NETS, Common Core, and state science and CTE standards  Defined computational thinking practices (CTP) and focal knowledge, skills, and abilities (FKSAs) that constitute them

12. © 2013 SRI International - Company Confidential and Proprietary Information PACT Project Accomplishments II  Developed and applied CTP design patterns to guide the development of assessments for ECS  Field testing assessments for ECS Units 1-4, and a summative assessment  Conducting think aloud interviews with ECS students

13. © 2013 SRI International - Company Confidential and Proprietary Information Menlo Park Headquarters SRI International 333 Ravenswood Avenue Menlo Park, CA 94025-3493 650.859.2000 Washington, D.C. SRI International 1100 Wilson Blvd., Suite 2800 Arlington, VA 22209-3915 703.524.2053 Additional U.S. and international locations www.sri.com Thank You! © 2012 Principled Assessment of Computational Thinking (PACT). Produced by the Center for Technology in Learning at SRI International with support from the National Science Foundation under contract numbers, CNS-1132232 and CNS-1240625 and CNS-0943507 to the University of Oregon. Any opinions, findings, conclusions, or recommendations expressed are those of the authors and do not necessarily reflect the views of the National Science Foundation.