Welcome! A-Team Session 4 Fall 2007. Overview for Today 1 Minute Message review Outcomes Feedback Methodology Review Assessment Steps  Sampling  Validity/Reliability.

Slides:



Advertisements
Similar presentations
Critical Reading Strategies: Overview of Research Process
Advertisements

Action Research Not traditional educational research often research tests theory not practical Teacher research in classrooms and/or schools/districts.
June 19, Proposal: An overall Plan Design to obtain answer to the research questions or problems Outline the various tasks you plan to undertake.
Research Methods in Psychology
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
The Purpose of Action Research
Research Methods Overview of quantitative and qualitative methods.
Reviewing and Critiquing Research
Reliability, Validity, Trustworthiness If a research says it must be right, then it must be right,… right??
SUMMER Internship Marks Allocation 2 Marking CriteriaMarks Allocated Workshops (4 in total)4*5 = 20 marks Academic Supervisor Attendance5 Report.
Qualitative Field Research Interviewing Focus Groups Ethnography Case Studies Grounded Theory Ethnomethodology.
SOWK 6003 Social Work Research Week 5 Measurement By Dr. Paul Wong.
Validity 4 For research to be considered valid, it must be based on fact or evidence; that is, it must be "capable of being justified."
WRITING A RESEARCH PROPOSAL
UOFYE Assessment Retreat
Chapter 14 Overview of Qualitative Research Gay, Mills, and Airasian
Proposal Writing.
Chapter 5 Formulating the research design
The Research Process Interpretivist Positivist
Chapter 9 Qualitative Data Analysis Copyright © 2013 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin.
Research Methods in Psychology (Pp 1-31). Research Studies Pay particular attention to research studies cited throughout your textbook(s) as you prepare.
Action Research March 12, 2012 Data Collection. Qualities of Data Collection  Generalizability – not necessary; goal is to improve school or classroom.
Analyzing Reliability and Validity in Outcomes Assessment (Part 1) Robert W. Lingard and Deborah K. van Alphen California State University, Northridge.
Research Methodology For IB Psychology Students. Empirical Investigation The collecting of objective information firsthand, by making careful measurements.
Research Methodology.
Performance Samples Patricia L. Sitlington, Ph.D. University of Northern Iowa.
Education 793 Class Notes Welcome! 3 September 2003.
CHAPTER III IMPLEMENTATIONANDPROCEDURES.  4-5 pages  Describes in detail how the study was conducted.  For a quantitative project, explain how you.
Evaluating a Research Report
Faustino Jerome G. Babate Nursing Research I Brokenshire College SOCSKSARGEN.
CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?
EVALUATION APPROACHES Heather Aquilina 24 March 2015.
Understanding Qualitative Assessment Skills Session #3 A-Team Training November 13, 2007.
Introduction to Research Deny A. Kwary Airlangga University
ScWk 242 Course Overview and Review of ScWk 240 Concepts ScWk 242 Session 1 Slides.
EDU 8603 Day 6. What do the following numbers mean?
Welcome! A-team Session 1 Fall Overview for Today Introductions What is assessment? Challenges & Concerns Map of A-Team Sessions & Outcomes Expectations.
Notes on Research Design You have decided –What the problem is –What the study goals are –Why it is important for you to do the study Now you will construct.
Further notes on methodology Indebted to Patton (1990)
September 2007 Survey Development Rita O'Sullivan Evaluation, Assessment, & Policy Connections (EvAP) School of Education, University of North Carolina-Chapel.
Basic Qualitative Strategies Cap Peck, Research Question Beginning with less specific research questions Start with interests and a sense of the.
Qualitative Data and Quantitative Data: Are they different?
Methods of Research and Enquiry Qualitative Case Study by Dr. Daniel Churchill.
Paper III Qualitative research methodology. Objective 1.3 To what extent can findings be generalized from qualitative studies.
Paradigms/Research Traditions “Instead of asking, how can this be true? We could ask, what if this were true? What then?” (Bochner, 2000, p. 267)
NSC 440 RESEARCH IN NURSING 4 UNITS DEPARTMENT OF NURSING SCIENCE FACULTY OF BASIC MEDICAL SCIENCES 1.
Performance Improvement Project Validation Process Outcome Focused Scoring Methodology and Critical Analysis Presenter: Christi Melendez, RN, CPHQ Associate.
Chapter Nine: Qualitative Procedures
Fundamentals of Governance: Parliament and Government Understanding and Demonstrating Assessment Criteria Facilitator: Tony Cash.
Onsite Quarterly Meeting SIPP PIPs June 13, 2012 Presenter: Christy Hormann, LMSW, CPHQ Project Leader-PIP Team.
Paper III Qualitative research methodology.  Qualitative research is designed to reveal a specific target audience’s range of behavior and the perceptions.
CRITICALLY APPRAISING EVIDENCE Lisa Broughton, PhD, RN, CCRN.
Research and Evaluation
Quality Assurance processes
Performance Improvement Project Validation Process Outcome Focused Scoring Methodology and Critical Analysis Presenter: Christi Melendez, RN, CPHQ Associate.
Reliability and Validity in Research
Introduction to educational research
Dealing with Validity, Reliability, and Ethics
Content analysis, thematic analysis and grounded theory
Performance Improvement Project Validation Process Outcome Focused Scoring Methodology and Critical Analysis Presenter: Christi Melendez, RN, CPHQ Associate.
Chapter 2 Sociological Research Methods
A Bit More About Qualitative Analysis
Trustworthiness in Qualitative Research
Analyzing Reliability and Validity in Outcomes Assessment Part 1
ACTION RESEARCH.
Integrating Outcomes Learning Community Call February 8, 2012
Introduction to Qualitative Research
Presenter: Kate Bell, MA PIP Reviewer
Analyzing Reliability and Validity in Outcomes Assessment
Ass. Prof. Dr. Mogeeb Mosleh
Presentation transcript:

Welcome! A-Team Session 4 Fall 2007

Overview for Today 1 Minute Message review Outcomes Feedback Methodology Review Assessment Steps  Sampling  Validity/Reliability  Analysis  Ethics/Reporting Take Home

Session II Overview of Outcome Design Creating Outcomes Service, Learning, & Development Session III Critiquing Outcomes 3 M’s Intro: Assessment Steps Session I Outcomes & Overview What is Assessment Assessment Language Comp Model & Foundation SALDOs Session IV Assessment Steps (cont) Sampling, Ethics, Reporting Session V Assessment Plans Session VI Assessment Plan Presentation Session Evaluation Foundation Session Outcomes Demonstrate an understanding of “A” language & literature Define and plan an assessment project for your department Increase Technical Skills— Perseus, Report Writing, Qual/Quan Perform 1 + assessment projects Assessment Resource for department and team

One Minute Message Any further questions or concerns about methods for your assessment projects?

Outcomes Feedback Remember the measurable piece—need to specify how you will measure in your outcome(s) Manageable: think about timelines Meaningful: great job on this one! Ideal order of assessment versus reality

Methodology Review Activity Is the word or phrase in reference to qualitative or quantitative methods?

Assessment Steps cont: So now you have a method, what’s next? Let’s talk about sampling… What’s a sample? Why is determining your sample important? What determines your strategy?

Quantitative Sampling Random sampling: Why is this important to quantitative assessments? Convenience sampling Stratified sampling Sample size Response rates

Qualitative Sampling Purposeful sampling: Why is this important to quantitative assessments? Snowball sampling Stratified sampling Sample size-contextual; about data saturation

Quantitative Analysis Frequency distributions (percentages, counts) Descriptive statistics (means) Useful in tracking reports, perceived responses- on Likert measurements) Inferential statistics- t Tests examines whether or not relationships are statistically significant (Ex: looking at differences in self concept scores between different types of disabilities)

Qualitative Analysis Data is rich and complex (Interviews, focus groups, documents, observations)  Coding-open, categorical, thematic  Looking for themes to emerge

Validity and Reliability Reliability- refers to the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials Validity-refers to the degree which a study accurately reflects or assess the specific concepts that the researcher is attempting to measure

What does R & V look like in Qualitative and Quantitative processes? For Quantitative:  R-concerned with consistency of measurements: Pilot tests  V-support for generalization of results to general population: Use random sample, correct sample size, and strong response rate. For Qualitative:  R-does it make sense; themes consistent with data collected: Member checking, peer examination, triangulation of data  V-how applicable to other situations for reader (can’t generalize): Use rich thick data, use of multiple sites

Ethics Accuracy—are you measuring what you want to measure? Quality—projects should be conducted with rigorous methods and high level of skill Confidentiality/Anonymity Informed Consent Role Conflicts Ownership/Dissemination of data

Reporting and Using Results Consider your audience  Language  Familiarity with subject matter  Time to read  Ability to act  Politics

Reporting and Using Results Format and Content  Executive Summary  Short Report (3-5 pages)  Full Report  Supplemental Reports  Meaningful Title  Recommendations  Visual Aids  Action/Implementation Plan

Final Thoughts Take Home Assignment One Minute Message