1/∞ CRESST/UCLA Towards Individualized Instruction Using Technology Gregory K. W. K. Chung Annual CRESST Conference Los Angeles, CA – January 22, 2007.

Slides:

Advertisements

Similar presentations

Professional Development on the Instructional Shift of Focus Lets Focus on Focus.

Advertisements

Common Core State Standards for Mathematics: Coherence Grade 7 Overview.

Common Core State Standards for Mathematics: Coherence

Department of Mathematics and Science

PARCC Tests: An Investment in Learning Test quality and rigor increase; Costs for states generally hold steady July 2013.

Advances in the PARCC Mathematics Assessment August

Understanding the Smarter BalanceD Math Summative Assessment

Got Data? Using your ACT tools to improve performance on all levels, moving students to College and Career Readiness.

MATH ASSESSMENT TEST OCMA May, HISTORY OF MAT Test originally developed in late 60’s.

A retrospective look : Action Research on the Fly:

1/∞ CRESST/UCLA Towards Individualized Instruction With Technology-Enabled Tools and Methods Gregory K. W. K. Chung, Girlie C. Delacruz, Gary B. Dionne,

C R E S S T / U C L A Improving the Validity of Measures by Focusing on Learning Eva L. Baker CRESST National Conference: Research Goes to School Los Angeles,

1/16 CRESST/UCLA Alternative Assessment for English Language Learners Christy Kim Boscardin Barbara Jones Shannon Madsen Claire Nishimura Jae-Eun Park.

Classroom Assessment Reliability. Classroom Assessment Reliability Reliability = Assessment Consistency. –Consistency within teachers across students.

Beverly Park Woolf University of Massachusetts/Amherst U.S.A

Experimental Design The Gold Standard?.

C R E S S T / U C L A Evaluating the Impact of the Interactive Multimedia Exercises (IMMEX) Program: Measuring the Impact of Problem-Solving Assessment.

COPYRIGHT WESTED, 2010 Calipers II: Using Simulations to Assess Complex Science Learning Diagnostic Assessments Panel DRK-12 PI Meeting - Dec 1–3, 2010.

DEVELOPING ALGEBRA-READY STUDENTS FOR MIDDLE SCHOOL: EXPLORING THE IMPACT OF EARLY ALGEBRA PRINCIPAL INVESTIGATORS:Maria L. Blanton, University of Massachusetts.

Moving the Red Queen Forward: Modeling Intersegmental Transition in Math Terrence Willett Director of Research.

Math Testing as it exists in S. C. now PACT PACT Grades 3 through 8 Grades 3 through 8 Standards-based Standards-based Used to measure academic progress.

P.E.R.T. Diagnostic Learning Pathways Math, Reading, Writing.

Pg 1 Design Research at GlassLab March 25 th, 2014.

Strategies for Efficient Scoring Jeanne Stone, UC Irvine PACT Conference October 22, 2009.

Evaluating Student Growth Looking at student works samples to evaluate for both CCSS- Math Content and Standards for Mathematical Practice.

LinearRelationships Jonathan Naka Intro to Algebra Unit Portfolio Presentation.

DePaul Assessment Certificate Culminating Project Assessment of STEM Center Algebra Initiative Learning Outcome Max Barry Department Assistant STEM Center.

Jan. 19 th DART STATEMENTS I can identify the weight of the Explore test on K- PREP. I understand my responsibility or role with the preparation.

1 / 27 California Educational Research Association 88 th Annual Conference Formative Assessment: Implications for Student Learning San Francisco, CA November.

Ensuring State Assessments Match the Rigor, Depth and Breadth of College- and Career- Ready Standards Student Achievement Partners Spring 2014.

Number Sense Standards Measurement and Geometry Statistics, Data Analysis and Probability CST Math 6 Released Questions Algebra and Functions 0 Questions.

Advances in the PARCC Mathematics Assessment August

Common Core and the Community College May 20, 2014.

Forum - 1 Assessments for Learning: A Briefing on Performance-Based Assessments Eva L. Baker Director National Center for Research on Evaluation, Standards,

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Online Assessment within.

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing V3, 4/9/07 SIG: Technology.

Validity Is the Test Appropriate, Useful, and Meaningful?

Achievethecore.org 1 Setting the Context for the Common Core State Standards Sandra Alberti Student Achievement Partners.

Baker ONR/NETC July 03 v.4  2003 Regents of the University of California ONR/NETC Planning Meeting 18 July, 2003 UCLA/CRESST, Los Angeles, CA ONR Advanced.

Students with Learning Disabilities Mathematics. Math Skills Development Learning readiness –Number instruction Classification, ordering, one-to-one correspondence.

Dr. John D. Barge, State School Superintendent “Making Education Work for All Georgians” Assessment for Learning Series Module 2: Understanding.

CEDAR RIDGE MIDDLE SCHOOL JANUARY 15, 2015 acos2010.wikispaces.com.

1/27 CRESST/UCLA DIAGNOSTIC/PRESCRIPTIVE USES OF COMPUTER- BASED ASSESSMENT OF PROBLEM SOLVING San-hui Sabrina Chuang CRESST Conference 2007 UCLA Graduate.

California Educational Research Association (CERA) San Francisco, CA – November 19, 2009 Gregory K.W.K. Chung Research Issues in Developing Games for Learning.

California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.

Teaching to the “Big Ideas”: Moving beyond the standards Terry P. Vendlinski UCLA Graduate School of Education & Information Studies National Center for.

Capturing Growth in Teacher Mathematical Knowledge The Association of Mathematics Teacher Educators Eleventh Annual Conference 26 January 2007 Dr. DeAnn.

CRESST ONR/NETC Meetings, July 2003, v1 17 July, 2003 ONR Advanced Distributed Learning Greg Chung Bill Bewley UCLA/CRESST Ontologies and Bayesian.

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Designing Instructional.

1 Assessing Student Understanding David Niemi UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards,

Acos2010.wikispaces.com. ACT Provides the Following:  a standards-based system of assessments to monitor progress toward college and career readiness.

Knowing What Students Know: The Science and Design of Educational Assessment Committee on the Foundations of Assessment Board on Testing and Assessment,

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing An Exploratory Study.

MATH INTERVENTION MATERIAL REVIEW: Vmath (VOYAGER) Peggy Cunningham.

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing V4, 1/18/07 Research.

Illinois State Board of Education A New Vision of Assessment: Texts Worth Reading, Problems Worth Solving, Tests Worth Taking Overview Presented by:

Assessing Math Looking closer at PARCC Task Types 2.

C R E S S T / U C L A UCLA Graduate School of Education & Information Studies Center for the Study of Evaluation National Center for Research on Evaluation,

Oregon’s Second Annual GED Summit Creating Promise, Designing Success Thank you to our sponsors:

Mater Gardens Middle School MATHEMATICS DEPARTMENT WHERE LEARNING HAS NO FINISH LINE ! 1.

SBAC-Mathematics November 26, Outcomes Further understand DOK in the area of Mathematics Understand how the new SBAC assessments will measure student.

Overview of CCSS and PARCC Cindy Weinrich. Common Core Curriculum State Standards for Math have Two Parts: Content Standards: What students need to know.

Department of Curriculum and Instruction Considerations for Choosing Mathematics Progress Monitoring Measures from K-12 Anne Foegen, Ph.D. Pursuing the.

Assessment of Learning 1

Dr. Robert H. Meyer Research Professor and Director

Milwee Middle School Math Night

CHAPTER 13: Assessing Mathematics Achievement

Eliminating testing through continuous assessment

What is a Performance Task

Assessment Literacy: Test Purpose and Use

Presentation transcript:

1/∞ CRESST/UCLA Towards Individualized Instruction Using Technology Gregory K. W. K. Chung Annual CRESST Conference Los Angeles, CA – January 22, 2007 UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing

2/∞ CRESST/UCLA Structure of Talk Problem statement Research questions Technical approach to diagnoses and prescription Study design

3/∞ CRESST/UCLA Problem Statement

4/∞ CRESST/UCLA Released NAEP Items NAEP correct Current Study Lo:.73 correct Hi:.87 correct NAEP correct Current Study Lo:.48 correct Hi:.69 correct NAEP correct Current Study Lo:.63 correct Hi:.87 correct.37 incorrect.13 incorrect.52 incorrect.31 incorrect.27 incorrect.13 incorrect.48 incorrect.52 incorrect.31 incorrect

5/∞ CRESST/UCLA Even Simpler Problems Lo:.36 correct Hi:.68 correct Lo:.73 correct Hi:.88 correct Lo:.68 correct Hi:.90 correct Lo:.60 correct Hi:.60 correct Lo:.56 correct Hi:.90 correct

6/∞ CRESST/UCLA Study Context Why pre-algebra? Pre-algebra provides students with the fundamental skills and knowledge that underlie algebra In fall 2004, 57% of CSU first-time freshmen needed remediation in mathematics (22,000 students) Remediation includes basic math, pre- algebra, geometry, and algebra By fall 2005, 3500 students did not complete remediation and were disenrolled

7/∞ CRESST/UCLA Study Context Tasked with developing instructional and assessment supports for pre-algebra Develop an approach to support rapid diagnosis and remediation of pre-algebra knowledge and skill gaps Develop assessment and instructional tools to support classroom instruction Test approach with middle school students Do it with technology, do it now

8/∞ CRESST/UCLA Research Questions

9/∞ CRESST/UCLA Research Questions What is the architecture for a diagnosis and remediation system? What are necessary components of such a system, how is diagnosis performed, how is remediation performed, how is the system validated, how is effectiveness measured?

10/∞ CRESST/UCLA Research Questions To what extent can a Bayesian network codify and capture the structure and properties of pre-algebra? This question addresses the use of BN for representing domain knowledge (i.e., pre- algebra) and automated reasoning.

11/∞ CRESST/UCLA Research Questions What are methods for developing “instructional parcels”? What is the structure of the student interaction, what is the format of the media delivery, what are techniques that can engage students, what are the scalability issues, what are the development bottlenecks?

12/∞ CRESST/UCLA Basic Approach Automated Reasoning (Bayes net of pre- algebra) Pre-algebra pretest adding fractions distributive property multiplying fractions Individualized instruction Individualized posttest

13/∞ CRESST/UCLA Domain Analysis Specify domain structure as a Bayes net Specify concepts and influences among concepts Attach math items (“observables”) to concepts to form scales Develop assessment items using a step- by-step derivation of a few complex problems Internally coherent Enables precise diagnoses

14/∞ CRESST/UCLA Pre-algebra Bayes Net definitions operations transformations common errors

15/∞ CRESST/UCLA Domain Sampling

16/∞ CRESST/UCLA Sample Items DP (OPER) CP-ADD (CE) AP-ADD (CE) DP (CE) DP (OPER) DP = distributive property CP-ADD = commutative property of addition AP-ADD = associative property of addition CE = common error OPER = operation

17/∞ CRESST/UCLA Instructional Design Issues

18/∞ CRESST/UCLA Study

19/∞ CRESST/UCLA Study Design and Procedure Pretest (1st occasion) Assign Instructional Parcels to S Score Tests (at UCLA) Generate Predictions Assign S to Conditions Occasion 1 Occasion 2 Computer Instruction Posttest Experimental Condition Posttest Control Condition Computer Instruction 2 days

20/∞ CRESST/UCLA Expected Relations PretestPosttest PretestPosttest PretestPosttest PretestPosttest Low Prior Knowledge High Prior Knowledge No Instruction Instruction Score

21/∞ CRESST/UCLA Results (L/M Prior Know.) a Wilcoxan signed rank test (non-parametric paired test). Scale InstructionNo Instruction neg.pos.tiepapa neg.pos.tiepapa Adding fractions2195<.01140ns Multiplying fractions ns Reducing fractions222ns103 Transformations5236< Distributive property ns

22/∞ CRESST/UCLA Results (L/M Prior Know.) a Wilcoxan signed rank test (non-parametric paired test). Scale InstructionNo Instruction neg.pos.tiepapa neg.pos.tie papa Commutative property of addition 115ns111 Commutative property of multiplication Associative property of addition 333ns115 Multiplicative inverse595ns140 Multiplicative identity6196<.01424ns

23/∞ CRESST/UCLA Discussion Preliminary results promising Individualization of instruction and assessment tractable Instructional parcels appear effective but more work needed Next steps Refine items and domain sampling End-to-end automation Continue gathering validity evidence

24/∞ CRESST/UCLA Questions

25/∞ CRESST/UCLA