Developing Valid Rubrics for Assessing Global Awareness and Global Perspective Dr. Stephanie Doscher Florida International University AAC&U General Education.

Slides:

Advertisements

Similar presentations

Aligning VALUE Rubrics to Institutional Purposes: Sharing Practical Approaches that Promote Consistent Scoring and Program Improvement Linda Siefert

Advertisements

2014/2015 English Course Offerings for Incoming Grade 11 Students Challenge Yourself!

Hitchcock ISD TELPAS Training. Rater Credentials Each teacher selected to rate an ELL must 1) have the student in class 2) be knowledgeable.

Outcomes Assessment. Many academic institutions measure the success of service- learning based on participation, number of hours, or estimated monies.

Session 3 The evidence should be credible & helpful. Implications: the assessments should – ◦ Be grounded in real-world applications, supplemented as.

Department of World Languages and ESL Northport-East Northport UFSD Mrs. Jennifer Nesfield District Chairperson for World Languages and ESL January 2014.

SLO Course Assessment in 5 Easy Steps Vivian Mun, Ed.D.

Arkansas Department of Education Literature Circles Mid-Level and High School.

Daniel Peck January 28, SLOs versus Course Objectives Student Learning Outcomes for the classroom describe the knowledge, skills, abilities.

TELPAS Grades K-1 Holistic Rating Training Spring 2010 Hitchcock ISD.

NSE Assessment (Session 2): The Final Story Project

Mara Manson, Ed.D Adelphi University March 28, 2014.

Independent STUDIES % March CLICK TO BEGIN

1 Assessing CORE Student Learning Outcomes Summer Assessment Institute August, 2005 Presented by Jerry Rudmann Coastline Community College.

Lamar State College-Port Arthur February 16 & 17, 2011.

Dallas Baptist University College of Education Graduate Programs

NCATE Institutional Orientation Session on PROGRAM REVIEW Moving Away from Input- based Programs Toward Performance-based Programs Emerson J. Elliott,

University Plan for the Assessment of Student Learning Spring 2006 Revisions Include: -Addition of Graduate School Learning Goals -Incorporation of recommendations.

Revised Requirement Course Proposal.  Change the Requirements for the Bachelor’s Degree from A to B. Successfully complete the General Education Requirements.

Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at New Jersey Institute of Technology Carol Siri Johnson Associate.

Principles of Assessment

Writing, Communication, and PBL Lima 2006 Using PBL to Improve Student Communication Skills Steve Bernhardt Tipsheets:

Spring 2012 Pilot Project Module Nine A New Texas Core Curriculum 1.

Assessment Tools. Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions.

Now That My Marking’s Done, Now What??? Brent Galloway Lori Stuber.

Assessment for Optimal Learning Tace Crouse Faculty Center for Teaching and Learning University of Central Florida.

PROFESSIONAL DEVELOPMENT PLAN WORKSHOP. What is the Professional Development Plan? The Professional Development Plan is a directed planning and evaluation.

Susan Agre-Kippenhan, Portland State University Professor, Art Department Evaluating the Effectiveness of Service Learning.

From the Beginning Planning for Impact in Writing Instruction, 6-12 Dr. Patti McWhorter

3/25/2011 Rubric Development. Overview  A Little About You  Overview A Quick Note about Validity Introduction of Rubrics How to Create a Rubric Using.

Advertising Public Relations Assessment Report Nicholson School of Communication.

* Research suggests that technology used in classrooms can be especially advantageous to at-risk, EL, and special ed students. (Means, Blando, Olson,

Session 2: Are We There Yet? Integrating Understanding by Design and Historical Thinking.

Professional growth Objectives 1)Pilot program 2)Familiarize/re-familiarize ourselves with new Standards and Indicators 3)Consider the importance of data.

26 March Independent STUDIES 100 % by Friday CLICK TO BEGIN Questions: Click Here 5 SOLUTIONS Directions: 1. Click on play. 2. Click to begin. 3. Click.

Everything You Need to Know for Spring 2010 S. Kashima.

 Student Growth Goals & Plan KASA Conference July 17, 2014.

Identification Evidence Options: The following are pieces of evidence that can be used for each identification area or category of giftedness. A minimum.

PBL for the 21 st century. Begin with the end in mind Knowledge of science, history, literature, languages, etc. Time management Strong work ethic Respectful.

Assessment Tools.

Authentic Assessment Kellie Dimmette CI Pretest on Evaluation Part I 1.C & D 2.B & C 3.T 4.Valid, reliable 5.T 6.T 7.T 8.A & B 9.C 10.B.

GCSE English Language and English Literature. The English Department follows the WJEC GCSE course.

College of Agriculture and Life Sciences WELCOME Assistant Professor P&T Workshop Defining Scholarship and P&T Portfolio Development April 22, 2015.

GCSE English Language and English Literature. The English Department follows the WJEC GCSE course.

Alternative Assessment Chapter 8 David Goh. Factors Increasing Awareness and Development of Alternative Assessment Educational reform movement Goals 2000,

What Your Program Needs to Know about Learning Outcomes Assessment at UGA.

Appropriate Use of Assessment Methods. Assessment Methods Utilized for “assessment for learning” ◦For promoting learning Applied for “assessment of learning”

Short Story Unit Planning My Unit Unit Summary This unit is about the short story. Mystery and suspense movies are some of my favorite kinds stories.

Module 3: Programmatic Assessment Strategies

COMMON CORE ASSISTANT TRAINING St. Pius X School Greensboro, NC.

Assessment strategies for improving students’ communication skills Dr Lata Gautam Department of Biomedical and Forensic Sciences,

FSA PARENT NIGHT. Agenda  What is the FSA?  FSA testing dates, Test Length, Sessions and Times  Achievement Levels  Promotion Requirements  FSA Test.

Today’s session will cover: 1)What is the We the People: the Citizen and the Constitution Level 1 and 2 curriculum? 2) What are "Mock Congressional Hearings"

Using Common Core Research Standards to Vertically Plan Research Throughout Your School Peter G. Mohn Glacier Peak High School 2016 NCCE Conference February.

College Career Ready Conference Participants will  Review components of the Grade 3 and the Grades 4 and 5 Condensed Scoring Rubric for Prose Constructed.

Welcome to the Central Campus Curriculum Night Intermediate Grades Reading

5 Year School Improvement Plan By 2015, enable students to demonstrate powerful thinking by systemically solving problems through analyzing.

The Development of Rubrics to Measure Undergraduate Students’ Global Awareness and Global Perspective: A Validity Study Stephanie Paul Doscher Department.

REVIEW Performance (Authentic) Assessments. Goes Beyond Testing Students must show they've gained certain knowledge and skills Usually focused on the.

Footer to be inserted here 1 ASSESSMENT of REFLECTION in Participation Units Theresa Winchester-Seeto & Marina Harvey with Debra Coulson & Jacqueline Mackaway.

Assessing Our Revised General Education Program

Helping students know what they know

Literacy Content Specialist, CDE

Implementing English II Benchmarks

FSA PARENT NIGHT.

Jillian Kinzie, Indiana University Center for Postsecondary Research

Dr. James W. Dottin Department Chair Business Administration

Welcome to On-level Language Arts!

Assessment Spring 2016.

Tennessee edTPA Conference

Presentation transcript:

Developing Valid Rubrics for Assessing Global Awareness and Global Perspective Dr. Stephanie Doscher Florida International University AAC&U General Education and Assessment Conference Boston, Massachusetts, March 1 st, 2013

Session Outline Global FIU Rubric anatomy Developing the rubrics Scorer training Rubric for rubrics Gathering evidence of validity and reliability

Text

FIUs Global Learning SLOs Global Awareness: knowledge of the interrelatedness of local, global, international and intercultural issues, trends, and systems Global Perspective: ability to develop a multi- perspective analysis of local, global, international, and intercultural problems Global Engagement: willingness to engage in local, global, international, and intercultural problem solving

Active Learning Strategies/ Performance Assessments Essay, story, or poem Research paper Literary analysis Book or article review Case study Speech Journal response Art exhibit Portfolio Musical composition Lab report Strategic planning In-class discussion Editorials Peer editing Poster presentation Video Podcast Mock trial Oral presentation Debate Role play Online discussion Blogs Advertising campaign Building a prototype Modeling Experiments Service learning

Developing the Rubrics Faculty learning community Pilot case studies, questions, scoring criteria Revisions based on response trends Pilot study Benchmark responses Anchor papers Faculty feedback, expert judge feedback Two field tests Scorer training Rubric language, response minimum

Scorer Training Full-time and adjunct faculty Pre-training packet Open discussion and review of cases, questions, and rubrics Norming session with anchor papers Sample scoring session, 10% of total papers

Validating the Rubrics Validity: Can the rubrics detect the differences in students development of global awareness and global perspective? Reliability: Can trained faculty raters agree on rubric scores 80% of the time or more?

Validating the Rubrics PretestTreatmentPosttest Global learning O 1 X 1 O Non-global learningO 2 X 2 O 2 Research Design

Data Collection s to chairs and potential faculty Pretestwithin first two weeks of class Posttestwithin last two weeks of class Trained faculty raters Two raters scored each question, third rater for discrepant scores

Results Inter-rater reliability Global awareness rubric Pretest (.89, p <.0001) Posttest (.95, p <.0001) Global perspective rubric Pretest (.92, p <.0001) Posttest (.91, p <.0001)

Results Validity – Global Awareness Rubric No significant main effects Post-hoc analysis: Groups differed significantly on pretest scores, p =.003 Global learning (M = 1.51) Non-global learning (M = 1.85) Significant interaction between global awareness pretest score and the treatment in predicting global awareness posttest score, p =.005

Global Awareness Rubric

Results Validity – Global Perspective Rubric No significant main effects Post-hoc analysis: Groups differed significantly on pretest scores, p =.003 Global learning (M =.90) Non-global learning (M = 1.2) Significant interaction between global perspective pretest score and the treatment in predicting global perspective posttest score, p =.005

Results

Interpretation of Results Reliability Both rubrics highly reliable Consistent with literature on rubric development and rater training Empirical support for structural validity

Interpretation of Results Validity Rubric detects differences between global learning and non-global learning students Rubric detects differences within the group of global learning students

Thank You! For more information, please contact me: Stephanie Doscher, Office of Global Learning Visit our web site: goglobal.fiu.edu