PATTON ON RIGOR. Michael Quinn Patton AEA 2010 SPEAKING TRUTH TO POWER 2.

Slides:



Advertisements
Similar presentations
Traits of an Effective Reader. What does a good reader do when they read? When good readers read, they read critically, they read deeply, they apply information.
Advertisements

Intelligence Step 5 - Capacity Analysis Capacity Analysis Without capacity, the most innovative and brilliant interventions will not be implemented, wont.
 To explain the NATURAL WORLD and how it got to be the way it is.  NOT merely to collect “facts” or describe.  Natural here means empirically sensible—that.
MB 450 Choosing a Research Strategy - 1
Interpretative Phenomenological Analysis (IPA)
Metadisciplinary Outcomes for Science Literacy (Can Assess Now by Standardized Concept Inventory) STUDENT WILL BE ABLE TO… 1. Define the domain of science.
The Essence of Critical Thinking the reasoned identification and evaluation of evidence to guide decision making analysis the form and content of evidence.
PHILOSOPHICAL PERSPECTIVES
CRTP Coursework Assignment Accounting for Leases Exposure Draft (ED/2013/6) Submission date: 20 January 2014 by 2:00 pm 1.
Research Philosophy Lecture 11th.
Analytical methods for Information Systems Professionals
Breadth First Search
Chapter 4 Understanding research philosophies and approaches
ISYS 3015 Research Methods ISYS3015 Analytical Methods for Information systems professionals Week 2 Lecture 1: The Research Process.
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Chapter 1 Explaining Behavior.
1 Writing the Research Proposal Researchers communicate: Plans, Methods, Thoughts, and Objectives for others to read discuss, and act upon.
An Evaluation of Microarray Visualization Tools for Biological Insight Presented by Tugrul Ince and Nir Peer University of Maryland Purvi Saraiya Chris.
SUNITA RAI PRINCIPAL KV AJNI
©2007 Prentice Hall Organizational Behavior: An Introduction to Your Life in Organizations Chapter 19 OB is for Life.
Educational Research: Action Research in Schools
Karin Hannes Centre for Methodology of Educational Research K.U.Leuven.
Allyn & Bacon 2003 Social Work Research Methods: Qualitative and Quantitative Approaches Topic 2: The Basics of Social Work Research Learn.
Curriculum Design for Assuring Learning in Business Education Session 2.
Reliability & Validity Qualitative Research Methods.
Looking at Understanding. What does it mean for this student to really understand Earth Science?
Critical thinking tools.
CRITICAL REFLECTION FOR CONTINUING PROFESSIONAL DEVELOPMENT: using the SOAP strategy to analyze pedagogical experience Padmini Boruah Department of English.
The Dissertation Advanced Higher Modern Studies. Marks Allocation The dissertation is worth 45 Marks (the exam is worth 90) How the 45 marks are allocated:
Qualitative Research.
Qualitative Research Approaches Research Methods Module Assoc Prof. Chiwoza R Bandawe.
Evidence-Based Practice Current knowledge and practice must be based on evidence of efficacy rather than intuition, tradition, or past practice. The importance.
Moving from Development to Efficacy & Intervention Fidelity Topics National Center for Special Education Research Grantee Meeting: June 28, 2010.
Qualitative Argues that meaning is situated in a particular perspective or context. Different people have different perspectives and contexts. There are.
Qualitative Methods for Health Program Evaluation
Chapter 1. Chemistry – study of the composition of substances and the changes they undergo Organic chemistry – study of carbon (C) containing substances.
WELNS 670: Wellness Research Design Chapter 5: Planning Your Research Design.
Understanding customer expectations and perceptions
Historical Thinking Skills
1 Mgt 540 Research Methods Section 2 Qualitative Research.
FOR 500 PRINCIPLES OF RESEARCH: PROPOSAL WRITING PROCESS
Ethnographic Research
Research studies that investigate the quality of relationships, activities, situations, or materials. Places a greater emphasis on holistic description.
Paradigms/Research Traditions “Instead of asking, how can this be true? We could ask, what if this were true? What then?” (Bochner, 2000, p. 267)
Qualitative Research. Narrative research How humans experience their lives How humans experience their lives Storied lives Storied lives Researchers construct.
VALUE/Multi-State Collaborative (MSC) to Advance Learning Outcomes Assessment Pilot Year Study Findings and Summary These slides summarize results from.
Qualitative Research EDUC 7741/Paris/Terry.
Ch 10 Methodology.
Evidence-Based Practice Evidence-Based Practice Current knowledge and practice must be based on evidence of efficacy rather than intuition, tradition,
Research for Nurses: Methods and Interpretation Chapter 1 What is research? What is nursing research? What are the goals of Nursing research?
Instilling rigor and imagination in analysis Analyst’s Roadmap Katherine Hibbs Pherson Mary C. Boardman 23 May 2012.
BBL 3403 RESEARCH METHODS IN LITERATURE DR. IDA BAIZURA BAHAR SLIDE H.
National Science Education Standards. Outline what students need to know, understand, and be able to do to be scientifically literate at different grade.
ANALYTICS IN HEALTHCARE PERFORMANCE ASSESSMENT METHODS.
Qualitative data analysis. Principles of qualitative data analysis Important for researchers to recognise and account for own perspective –Respondent.
EDU 626 Students Guide - snaptutorial.com snaptutorial.com For More Tutorials
Evaluation Requirements for MSP and Characteristics of Designs to Estimate Impacts with Confidence Ellen Bobronnikov March 23, 2011.
Unit 1 Question Style: USA
Planning an Applied Research Project
10.4 Qualitative Research Case Studies Ms. Binns.
Scholarship of Teaching and Learning Institute August 22, 2017
Academic Rubric Slides
IB Environmental Systems and Societies
KP7 DBQ Practice The Scopes Trial.
UNCLASSIFIED Intelligence Community Professional Education - The Canadian Perspective.
Qualitative Research.
BBL 3403 RESEARCH METHODS IN LITERATURE
HCI Evaluation Techniques
Asking the Right Questions
AP U.S. History Exam Details
Literature review The Nuts and bolts.
Presentation transcript:

PATTON ON RIGOR

Michael Quinn Patton AEA 2010 SPEAKING TRUTH TO POWER 2

Beyond method as the focus of evaluation quality and rigor

Rigor Attribute Model Rigor is not understood by assessing deviations of process, but rather, by assessing the contextual sufficiency of many different aspects of the analytic process. Cognitive Systems Approach: While professional intelligence analysts can make perceptive assessment about quality of an analysis based on product quality, these perceptions are apt to change with insight into the analytic process.

Rigor Attribute Model 1.Information Search relates to the depth and breadth of the search process used in collecting data. 2.Hypothesis Exploration -- extent to which multiple hypotheses are considered in explaining data. In a low-rigor process there is minimal weighing of alternatives

Rigor Attribute Model 3. Information Validation details the level at which information sources are corroborated and cross-validated. 4. Stance Analysis is the evaluation of data with the goal of identifying the stance or perspective of the source and placing it into a broader context of understanding.

Rigor Attribute Model 5. Sensitivity Analysis considers the extent to which the analyst considers and understands the assumptions and limitations of their analysis. 6. Specialist Collaboration describes the degree to which an analyst incorporates the perspectives of domain experts into their assessments.

Rigor Attribute Model 7. Information Synthesis refers to how far beyond simply collecting and listing data analysts went in their process. 8. Explanation Critique is a different form of collaboration that captures how many different perspectives were incorporated in examining the primary hypotheses. (Zelik, Patterson & Woods, 2007).

Quality of Thought Evaluation quality as critical, rigorous thinking vs. Mechanistic, methods-driven rigor

10 Minneapolis bridge collapse, Aug 1, 2007 Quality matters. Evaluation use matters.