Focus Group Methodology  Five focus groups science educators (n = 38)  K-5, 6-12 (inservice and preservice group), undergraduate faculty (two groups)

Slides:



Advertisements
Similar presentations
Writing an IRA/NCATE SPA Report. IRA Requirements Programs must have: –Minimum of 24 credit hours of reading/literacy courses aligned with IRA Standards.
Advertisements

Strategies to Measure Student Writing Skills in Your Disciplines Joan Hawthorne University of North Dakota.
Action Research Not traditional educational research often research tests theory not practical Teacher research in classrooms and/or schools/districts.
Evaluating Search Engine
A framework for thinking about mechanisms Range of possible implementation mechanisms Some criteria for assessing various mechanisms Holding tank description.
© Tefko Saracevic, Rutgers University 1 EVALUATION in searching IR systems Digital libraries Reference sources Web sources.
Retrieval Evaluation. Brief Review Evaluation of implementations in computer science often is in terms of time and space complexity. With large document.
Dogan Seber, PhD San Diego Supercomputer Center University of California, San Diego I. DLESE Library II. DISCOVER OUR EARTH Earth Science Resources for.
Soo Young Rieh School of Information University of Michigan Information Ethics Roundtable Misinformation and Disinformation April 3-4, 2009 University.
CLASSROOM ASSESSMENT FOR STUDENT LEARNING
Chapter 7: Evaluating Educational Technology and Integration Strategies Lindsey Blome, Jared Petz, Andrea Branam, Jessica Crowden, Amera Covert.
Digital Libraries: New Tools for ScienceTeaching and Learning.
Making the Most of Digital Learning Resources for STEM with NSDL 2010 Robert Noyce Teacher Scholarship Program Conference Washington DC July 8-9, 2010.
What we learned while building DLESE Katy Ginger Metadata Architect, Meteorologist, Instructional Designer.
Resources for Teaching Teachers Earth Science Content and Pedagogy The Association for Science Teacher Education Rusty Low Shelley Olds January 2006.
South African Education Portal
For each of the Climate Literacy and Energy Literacy Principles, a dedicated page on the CLEAN website summarizes the relevant scientific concepts and.
Enrich Social Studies Exit Project with Technolgoy Using TrackStar for Exit Project Research iLearn Social Studies Session 3 of 8.
LIS510 lecture 3 Thomas Krichel information storage & retrieval this area is now more know as information retrieval when I dealt with it I.
Ensemble Computing in the National Science Digital Library (NSDL)
Creating and Operating a Digital Library for Information and Learning– the GROW Project Muniram Budhu Department of Civil Engineering & Engineering Mechanics.
Education and Outreach Goals Increase Audience Awareness Facilitate Audience Engagement Along a User-Contributor Continuum Support Audience Needs.
Differential Effects of Participatory Evaluation in a National Multi-site Program Evaluation Frances Lawrenz University of Minnesota.
UTSA Symposium 2006 Assignments as Assessment Michelle S. Millet Information Literacy Coordinator Trinity University.
Different approaches and experiences at the University of Parma (Italy) Embedding information literacy into subject learning Fabrizia Bevilacqua Monica.
Metadata Lessons Learned Katy Ginger Digital Learning Sciences University Corporation for Atmospheric Research (UCAR)
How to Write a Syllabus for MYP. The easy stuff IB Course Title Level (IB year and traditional grade level) General course of study (if different from.
William E. Dugger, Jr., DTE Shelli Meade Lisa Delany The Complete Picture STL and AETL.
Dr. Lesley Farmer California State University Long Beach
The Digital Library for Earth System Science: Contributing resources and collections Meeting with GLOBE 5/29/03 Holly Devaul.
CT 854: Assessment and Evaluation in Science & Mathematics
The National Science Digital Library & Shibboleth.
Data Demo: Atmospheric Visualization Collection Christopher Klaus 05/18/05.
COSEE California Communicating Ocean Sciences Session 4: Building Towards Inquiry.
1 A Very Large Digital Library Technology Demonstration William Y. Arms Cornell University.
The Digital Library for Earth System Science: Contributing resources and collections GCCS Internship Orientation Holly Devaul 19 June 2003.
Mathematics and Science Partnerships: Summary of the FY2006 Annual Reports U.S. Department of Education.
DLESE Metadata Story Katy Ginger Metadata Architect DLESE Program Center Session: Metadata stories – The Creation and Management of Metadata.
Jim Dorward Sarah Giersch Kaye Howe Rena Janke Mimi Recker Andy Walker NSF Awards: NSDL ;TPC Using Online Science & Math Resources in Classrooms.
Chapter 8 Evaluating Search Engine. Evaluation n Evaluation is key to building effective and efficient search engines  Measurement usually carried out.
Session 4 Performance-Based Assessment
1 Understanding Cataloging with DLESE Metadata Karon Kelly Katy Ginger Holly Devaul
Resources and Reflections: Using Data in Undergraduate Geosciences Cathy Manduca SERC Carleton College DLESE Annual Meeting 2003.
After testing users Compile Data Compile Data Summarize Summarize Analyze Analyze Develop recommendations Develop recommendations Produce final report.
Information Literacy Online: p.r.n. Information Literacy Standards Delivery and Design Face to Face Partnerships Tutorials E-struction Assessment.
Evaluating Educational Resources W301. What are Educational Resources? There are many kinds of resources available to you: –Productivity Resources are.
COLLABORATING TO BUILD THE DIGITAL WATER EDUCATION LIBRARY DWEL Collaborative Projects in the NSDL: Institutional Collaborations Bryan L. Aivazian DWEL.
Education Transform Resources
Cataloging and Searching by Standards in DLESE Metadata Workshop 9-10 March 2004 Holly Devaul.
Auburn School District Student Growth Goals June
1 Annotation Framework March Terminology CV - abbreviation for controlled vocabulary CRS - Community Review System (a collection within DLESE)
Project VIABLE - Direct Behavior Rating: Evaluating Behaviors with Positive and Negative Definitions Rose Jaffery 1, Albee T. Ongusco 3, Amy M. Briesch.
DLESE DATA SERVICES: FACILITATING THE DEVELOPMENT AND EFFECTIVE USE OF EARTH SYSTEM SCIENCE DATA IN EDUCATION Tamara Shapiro Ledley Ben Domenico, Michael.
1 Using DLESE: Finding Resources to Enhance Teaching Shelley Olds Holly Devaul 11 July 2004.
How can I use a digital library to support my teaching? Find good resources to enhance existing curriculum  Search special collections aimed at your interests.
NSDL: A New Tool for Teaching and Learning.
The Digital Library for Earth System Science
Assessment Information
Using Data in Undergraduate Science Classrooms
What Makes a Good K-12 Resource
Evaluating Materials & Activities
The National Science Digital Library (NSDL)
Method Results Discussion
Using Data Workshops etc.
The Instructional Architect as a context for studying resource reuse
The Instructional Architect as a context for studying resource reuse
Steps Toward A National Digital Library for Science, Mathematics, Engineering and Technology Education Alice Agogino, Gene Klotz, Len Simutis, Flora McMartin,
AVC User Communities.
Presentation transcript:

Focus Group Methodology  Five focus groups science educators (n = 38)  K-5, 6-12 (inservice and preservice group), undergraduate faculty (two groups)  Instructed to provide advice to DLESE and NSDL on policies, priorities, and best practices for building useful digital collections for classroom use  Each focus group lasted between 60 and 90 minutes: 1. Briefed on DLESE and NSDL, and demo of DLESE collection 2. Briefed on relationships between the World Wide Web, the DLESE Broad Collection, and the DLESE Reviewed Collection 3. Explanation of the ‘3Ps,’ how to develop importance categories 4. Each person directed to an individual computer and given minutes to evaluate 4 educational web sites 5. Getting to resources from bookmarks, not DLESE 6. Group discussion - contribute the positive and negative criteria, then 3P’s

Evaluation Rubric

Results (1) - Policies, Priorities, Practices  Scientific accuracy (policy, all groups but k-5)  Complement, don’t replicate traditional materials (priority, all groups)  Libraries should label biased sites in the item-level metadata (all groups except K-5; either policy or priority)  ‘I know that you know’  Library policies or priorities should limit the accessioning of sites with advertising (all groups)  Discrete advertising, preferably related to scientific or educational products, that does not detract from learning (i.e. no pop-ups)  Frustrated with current state of web resource design (all groups)  All sites had usability problems, but particularly ambiguously rated (0-1) sites  Presence of Distractions - Student time on task a concern across all grade levels

Results (2) - Calibration, Added Value  Gestalt ratings - people agree more on ‘good’ resources (lower stdev) than ‘bad’ (higher stdev)  Consistency between rankings of teachers and DLESE collections expert  Remarkable consistency in recommendations across middle, high, and undergrad groups  K-5 have very different concerns  Participants expect library metadata to add value over the content of resources, not just to summarize  High expectations for NSF-funded projects  Noting bias and classroom uses (but how?)  Accurate and more precise designation of grade levels (and standards)  K-5: reading level often assumed to be too high  Many: grade ranges too broad

Recommendations - from JCDL 1. Shift from growing the collection quickly to growing it selectively  Higher standard of selection criteria for collections accessioning and funded cataloging efforts (distractions, advertising, interactivity) 2. Tightly couple collections development and library use  Co-development model provides context for design improvements, assessment, generating contextual information and reviewing  Library – school district – content provider partnerships 3. Reviewing  Identifying the ‘problematic’ is as important as the ‘good’

Designing for “Quality”  Relevance of search results  Metadata descriptions  Resources  Annotations and reviews Library systems, policies, work processes  Role of mediating tools such as digital libraries  Models: unaided human mind informational site  Predictive and Evaluative Judgments (Rieh 2002) Library Resource Classroom use