Internet based assessments via collaborative problem solving: The Assessment and Teaching of 21 st Century Skills (ATC21S™) Project Patrick Griffin Executive.

Slides:



Advertisements
Similar presentations
WV High Quality Standards for Schools
Advertisements

Thinking Skills and Personal Capabilities Unit 1
TWS Aid for Supervisors & Mentor Teachers Background on the TWS.
Intelligence Step 5 - Capacity Analysis Capacity Analysis Without capacity, the most innovative and brilliant interventions will not be implemented, wont.
Department of Education, Employment and Workplace Relations
Purpose of Instruction
PORTFOLIO.
The New Basics Project. Qld State Education An integrated framework for curriculum, pedagogy and assessment that defines essential areas of learning,
1 © 2006 Curriculum K-12 Directorate, NSW Department of Education and Training Implementing English K-6 Using the syllabus for consistency of teacher judgement.
Dr Jim Briggs Masterliness Not got an MSc myself; BA DPhil; been teaching masters students for 18 years.
Fit to Learn Using the Employability Skills Framework to improve your performance at College The Employability Skills Framework has been developed by business.
Marzano Art and Science Teaching Framework Learning Map
Consistency of Assessment
CHAPTER 3 ~~~~~ INFORMAL ASSESSMENT: SELECTING, SCORING, REPORTING.
Developmentally Appropriate Practices (DAP)
Meaningful Learning in an Information Age
INACOL National Standards for Quality Online Teaching, Version 2.
Matt Moxham EDUC 290. The Idaho Core Teacher Standards are ten standards set by the State of Idaho that teachers are expected to uphold. This is because.
Assessment for teaching Presented at the Black Sea Conference, Batumi, September 12, Patrick Griffin Assessment Research Centre Melbourne Graduate.
Macquarie University Library1 LAMS, e-learning & Information Literacy: possibilities & practicalities Margaret Wright, Macquarie University Heather Cooper,
Developing Effective Questioning In Teaching Games For Understanding (TGfU) Pearson & Webb, 2008.
Existing Frameworks: UNESCO Integration in stages The introduction and use of ICT in education proceeds in broad stages that may be conceived as a continuum.
School Leaders Professional Learning for School Leaders: The Principal’s Role in School Transformation Cynthia Mruczek Rich Barbacane April 19, 2011.
Intel ® Teach Essentials ICT in the Classroom Conference St John's College, Johannesburg 5-7 July 2011 Workshop by Claire Dean Senior Trainer SchoolNet.
Intel Teach Elements Collaboration in the Digital Classroom Charity I. Mulig First Webinar Session October 18, :00 – 9:30 pm.
Thinking Actively in a Social Context T A S C.
Assessment of Higher Education Learning Outcomes (AHELO): Update Deborah Roseveare Head, Skills beyond School Division Directorate for Education OECD 31.
Home, school & community partnerships Leadership & co-ordination Strategies & targets Monitoring & assessment Classroom teaching strategies Professional.
Interstate New Teacher Assessment and Support Consortium (INTASC)
Domain 1: Preparation and Planning. ElementUnsatisfactoryBasicProficientDistinguished Knowledge of content and the structure of the discipline In planning.
A Framework for Inquiry-Based Instruction through
Leadership: Connecting Vision With Action Presented by: Jan Stanley Spring 2010 Title I Directors’ Meeting.
Purpose of study A high-quality computing education equips pupils to use computational thinking and creativity to understand and change the world. Computing.
Educator Effectiveness Academy STEM Follow-Up Webinar December 2011.
Assessing Collaborative Problem Solving Martin Ripley.
Dr. Jana Jagodick Polytechnic of Namibia, 2012 Project Management Chapter 3 Project Management for Strategic Goal Achievement.
Bruce White Ruth Geer University of South Australia.
Presidential National Commission on Information Society and Development (PNC on ISAD)  Government commitment to information society Accelerated Shared.
1 Cisco/Intel/MS Education Sync Meeting The Cisco-Intel-Microsoft 21 st Century Skills Assessment Project Robert Kozma, Kozmalone Consulting Project Consultant.
Designing Local Curriculum Module 5. Objective To assist district leadership facilitate the development of local curricula.
Marzano’s Instructional Design Questions Spencer Bragan Lois Neumann University of New England EDU 751 October 6, 2013.
Training and Developing a Competitive Workforce 17/04/2013.
‘Sujala’ Karnataka Watershed Development Programme Developing the Training Strategy Introduction for the “Brainstorming Workshop 1st of October 2003.
Programming the New Syllabuses (incorporating the Australian Curriculum)
Summative vs. Formative Assessment. What Is Formative Assessment? Formative assessment is a systematic process to continuously gather evidence about learning.
Student Name Student Number ePortfolio Demonstrating my achievement of the NSW Institute of Teachers Graduate Teacher Stage of the Professional Teacher.
Queen’s Management & Leadership Framework
LEARNER CENTERED APPROACH
Session Objectives Analyze the key components and process of PBL Evaluate the potential benefits and limitations of using PBL Prepare a draft plan for.
Intel ® Teach Program International Curriculum Roundtable Programs of the Intel ® Education Initiative are funded by the Intel Foundation and Intel Corporation.
Problem-Solving Approach of Allied Health Learning Community.
Consultant Advance Research Team. Outline UNDERSTANDING M&E DATA NEEDS PEOPLE, PARTNERSHIP AND PLANNING 1.Organizational structures with HIV M&E functions.
Intro to Outcomes. What is “Outcomes”? A. a statewide initiative aimed at improving learning and accountability in education B. a standing SFCC committee.
Deep Dive and Practice. Standard 2d. Adapts teaching for the benefit of students with special needs 2.
Kathy Corbiere Service Delivery and Performance Commission
Google Earth INTEGRATING GLOBAL THINKING. Why Use Virtual Tours? Flexible Tool: History, Science, Math, English, etc. An Interactive Way to Explore Supports.
© Crown copyright 2008 Subject Leaders’ Development Meeting Spring 2009.
Structuring Learning. Agenda Structuring learning. Structuring lab sessions. Engagement. Critical Thinking. Ideas for structuring learning. Activity.
Supplemental Text Project Kenn Ward EDL 678 Dr. Pfennig June 2013.
Chapter 6 Assessing Science Learning Updated Spring 2012 – D. Fulton.
Authentic service-learning experiences, while almost endlessly diverse, have some common characteristics: Positive, meaningful and real to the participants.
ACS WASC/CDE Visiting Committee Final Presentation Panorama High School March
An Introduction to the Colorado Assessment Standards Reading - Writing - Communicating.
ATC21S Assessment and Teaching of 21st Century Skills The University of Melbourne Cisco Intel Microsoft Development and adoption of project.
Deep Dive and Practice.
Iowa Teaching Standards & Criteria
california Standards for the Teaching Profession
Bellwork: Student Engagement Chart
Developmentally Appropriate Practices (DAP)
Presentation transcript:

Internet based assessments via collaborative problem solving: The Assessment and Teaching of 21 st Century Skills (ATC21S™) Project Patrick Griffin Executive Director, ATC21S

Patrick Griffin, University of Melbourne, Executive DirectorATC21S Michael Stevenson, Cisco Vice President - Global Education, Former ATC21S Board Chair (2009–2010) Shelly Esque, Intel Vice President - Legal and Corporate Affairs, Current ATC21S Board Chair (2011–2012) Anthony Salcito, Microsoft Vice President - Education, Former ATC21S Board Chair (2010–2011) Esther Care, University of Melbourne, ATC21S International Research Coordinator Ministerial representatives from… –Australia: –Finland: –Singapore –USA: –Costa Rica: –Netherlands: EXECUTIVE BOARD PROJECT GOVERNANCE

ADVISORY BOARD Patrick Griffin, University of Melbourne, Executive Director ATC21S (Chair) Andreas Schleicher, Organization for Economic Co-operation and Development (OECD) Seamus Hegarty, International Association for the Evaluation of Educational Achievement (IEA) Irina Bokova (Director General UNESCO) Ray Adams, Technical Director PISA Marc Durando, European Schoolnet Esther Care, University of Melbourne, ATC21S International Research Coordinator Stuart Elliott, National Academy of Sciences David Forster, International Testing Commission Robin Horn, World Bank Eugenio Eduardo Severin, Inter-American Development Bank National project managers from each founder country and associate country Task force member from each company: Katrina Reynan, Director, Cisco Martina Roth, Director, Intel Greg Butler, Director, Microsoft

 Traditional assessments may not be suited to measure many 21 st century skills  Goal is to develop new assessment approaches matched to new C21 skills and to advise systems, schools and teachers on the use of assessment data to help students develop higher order performances Focus: the Assessment and Teaching of 21 st Century Skills Phase 1 Conceptualise C21 skills and education output needs Phase 2 Skill Identification and hypotheses Phase 3 Development and coding via coglabs Phase 4 Pilot studies and trials Phase 5 Dissemination scale and policy

CONCEPTUALIZING THE SKILLS Assemble experts Assemble experts Define methods Define methods Explore practical and technical needs in the classroom Explore practical and technical needs in the classroom Create a new framework Create a new framework Five working groups were established:  Defining 21st-century skills: Ms. Senta Raizen, WestEd  Methodological issues: Dr. Mark Wilson, University of California, Berkeley  Technological issues: Dr. Beno Csapo, University of Szeged, Hungary  Classrooms and formative evaluation: Dr. John Bransford, University of Washington, and Dr. Marlene Scardamalia, University of Toronto  Policy frameworks and new assessments: Dr. Linda Darling-Hammond, Stanford University Five working groups were established:  Defining 21st-century skills: Ms. Senta Raizen, WestEd  Methodological issues: Dr. Mark Wilson, University of California, Berkeley  Technological issues: Dr. Beno Csapo, University of Szeged, Hungary  Classrooms and formative evaluation: Dr. John Bransford, University of Washington, and Dr. Marlene Scardamalia, University of Toronto  Policy frameworks and new assessments: Dr. Linda Darling-Hammond, Stanford University

21ST-CENTURY SKILLS DEFINED ATC21SATC21S started by internationally defining 21st-century skills as four broad categories. ATC21S

Collaborative problem-solving ATC21S Assessment and Teaching of 21st Century Skills The University of Melbourne Cisco Intel Microsoft

Components Collaborative problem solving Social skillsParticipation Perspective taking Social regulation Cognitive skillsTask regulation Knowledge building

Participation skills over levels

Perspective taking skills

Social regulation skills

Task regulation skills

Knowledge building skills

Developin assessment Tasks for ATC21S Founder countries Associate and founder countries Draft -- Concept check for reality Panel – cognitive laboratory for codes Pilot – for administration Trials – for calibration

Tasks and Assignments to assess ICT Literacy Poetry (Graphic Organizers, Creating and Listening to Audio and video) Arctic Trek (Collaborative Notebook, Information Foraging)

ATC21S Twenty first century skills

Collecting the Data

Coding and scoring

Cognitive Behaviour U2L001A = presence of chat before any action Social Behaviour U2L00 4A = All positions hav been covered (providing player had access to 3 balls) U2L006A = sewuential placement of balls -6 combinations LMRLMR RMPRML RMLLMR LMRRML LLMMRR RRMMLL

Calibration and Interpretation

ElementAmplificationLowMiddleHigh Participation 1.Action Activity within environment No or very little activity in environment Activity in scaffolded context Activity in scaffolded and unscaffolded contexts 2.Interaction Interacting with, prompting and responding to the contributions of others Acknowledges communication Responding to cues in communication Initiating and promoting interaction or activity 3.Task completion Undertaking and completing a task or part of a task individually Maintains presence only (lurking) Identifies and attempts the task Perseveres in task as indicated by repeated attempts or multiple strategies Perspective taking 4.Responding adaptive responsiveness Ignoring, accepting or adapting contributions of others Contributions or prompts from others are ignored Contributions or prompts from others are taken into account Contributions or prompts of others are adapted and incorporated appropriately 5.Audience awareness Mutual modelling Awareness of how to adapt behaviour to increase suitability for others Contributions are not tailored to participants Contributions are modified for recipient understanding in the light of feedback Contributions are tailored to recipients (audience design) Social regulation 6.Negotiation Achieving a resolution or reaching compromise No attempts to negotiate joint understanding Comments on differences in perspective Negotiates through differences in perspective 7.Metamemory Self concept Recognising own strengths and weaknesses Notes own performance Comments on own performance in terms of appropriateness or adequacy Comments on own performance in terms of appropriateness or adequacy in the context of the task 8.Transactive memory Recognising strengths and weaknesses of others Notes performance of others Comments on performance of others in terms of appropriateness or adequacy Comments on performance of others in terms of appropriateness or adequacy in the context of the task 9.responsibility Initiative Assuming responsibility for ensuring aspects of task are completed by the group Undertakes activities largely independently of others Completes activities and reports to others Assumes group responsibility as indicated by use of second person plural or accepting others’ contribution 11. Resource managementManaging resources or people to complete a task Uses resources (or directs people) without consultation. Suggests that people or resources be used in part of a task Allocates people or resources to a task through to completion

ElementAmplificationLowMiddleHigh Knowledge Building 20. Knowledge acquisitionFollow path to gain knowledge Acquires knowledge as a result of being given it directly Deliberate single actions to acquire knowledge Knowledge acquired through multiple purposeful actions 12. Rules “If …then” (planning and executing) Formulating a course of action to address a problem or task Activity is undertaken with little or no prior formulation for a course of action Identifies short sequences of actions for a specific task Identifies potential multiple sequence routes for a complex task 19. Relationships (representing and formulating) Making connections between elements of knowledge Focused on (acts/shares) isolated pieces of information Building on input and information from others Integrates and synthesises of multiple pieces of information 18. Hypothesis “what if…” (reflecting and monitoring Changing from one line of reasoning or course of action to another as information or circumstances change Maintains a single line of approach tries multiple options in light of new information or lack of progress Reconstructs and reorganizes understanding of the problem in light of ne information or opinion. Task Regulation 15. Collecting ElementsExplore and understand Does not recognize the need for further information Identifies the need for specific information related o immediate activity Identifies need for varied information related to multiple activities. 13. Systematicity Implementing possible solutions to a monitoring progress Trial and error hypothesis testing in an unorganized sequence of solution attempts Forward search through a problem space with an organized sequence of solution attempts Forward and backward search through a problem space with reflective solution attempts 17. Tolerance for ambiguity Accepting ambiguous situations and exploring options within these Maintains only a presence in situations where there is ambiguity notes ambiguity and suggests options Explores the problem space 14. Organising (Problem analysis) Analysing and defining a problem in familiar language (i.e. Making the problem more manageable and meaningful) Problem is stated as initially represented either explicitly or implicitly Problem is divided into sub problems Problem is divided into sub problems and their inter dependence is recognised

Dimensionality

- cognitive

social

Developmental Progressions Social Strand Cognitive Strand Level 5- Cooperation, Shared Goals and Appreciation Level 5 Refined strategic application & problem solving Level 4 Mutual Commitment Level 4 Systematic and methodical Level 3 Resolving Differences Level 3 Strategic Planning Organising & Executing Level 2 Awareness of Partnership Level 2 directed approach Systematic Trial & Error Level 1 Clarification and Independence Level 1 Trial and Error Exploration

Feedback and Reporting

Reports

Reporting to Students Learning Readiness Real time reports will be available, based on empirically developed learning progressions. They will identify a point of readiness to learn for each student. This type of report will be linked to teaching interventions associated with readiness to learn as indicated by the black bar in the spine of the chart.

Instructional groups

Teacher development

Professional development modules 1.Defining and Assessing 21st Century Skills 2.Using a Developmental Model 3.ATC21S Assessments: Getting Started 4.Interpreting Reports 5.Teaching and Learning 21st Century Skills

The System An integrated system – Developmental progressions – standards referenced – Assessment tasks with automatic scoring – Teaching intervention and professional development – Reports for students, teachers and systems Next steps Access and Use Validation studies Expanded resources – mainstream curriculum Other 21 st century skills ATC21S Assessment and Teaching of 21st Century Skills The University of Melbourne Cisco Intel Microsoft

Transfer and Utility To what degree will 21st century skills such as collaborative problem solving or ICT Literacy in Learning Networks facilitate student learning? –Are these skills teachable and learnable? –Will the skills enhance learning in the classroom? To what degree might enhanced 21 st century skills contribute to adaptiveness of graduate students to the workforce?

ATC21s New Users?

Follow up!

Implications

Conventional Assessment Correctness if the important part of the response Response can indicate reasoning ability of a specific level Difficult to interpret various incorrect responses Additipnal data is lost in the focus on the correct response

Future Applications

Task design I.Design tasks to capture specific components of cognitive ability I.working memory II.Processing speed III.Pattern recognition IV.Systematicity’ II.Background data capture allows more user friendly and engaging asssessments I.Reduce test anxiety II.counteract teaching to the test III.Capture multiple variables IV.Reduce test time need V.Increase assessment efficiency VI.Better, richer information for teachers III.Applications in serious game industry

Roll Out and Dissemination

What is in the system? Two models - cloud and portable Prototype assessment tasks Developmental progressions Reporting and feedback for schools, teachers and students Professional development modules Specifications for developers

Portable and Cloud Systems Portable versionCloud version Local adaptationsCentralised control/management Decision-making autonomyCentralised decision-making Customisation enabled Embed or link within existing systems Local adaptations Addition of tasks No customisation Local support structureCentralised support structure Flexible registration systemRigid registration system Flash Less susceptible to differences across browsers Not currently supported on iPad* HTML 5 More susceptible to differences across browsers

PISA 2015 Collaborative problem solving to be tested in OECD countries in PISA will test human to computer interaction Collaborative Problem solving involving human to human maybe in 2018