Instrumentation: Reliability Measuring Caring in Nursing

Slides:



Advertisements
Similar presentations
Questionnaire Development
Advertisements

Reliability IOP 301-T Mr. Rajesh Gunesh Reliability  Reliability means repeatability or consistency  A measure is considered reliable if it would give.
MEASUREMENT: RELIABILITY Lu Ann Aday, Ph.D. The University of Texas School of Public Health.
Topics: Quality of Measurements
Types of Reliability.
MEASUREMENT CONCEPTS © 2012 The McGraw-Hill Companies, Inc.
© McGraw-Hill Higher Education. All rights reserved. Chapter 3 Reliability and Objectivity.
Chapter 5 Reliability Robert J. Drummond and Karyn Dayle Jones Assessment Procedures for Counselors and Helping Professionals, 6 th edition Copyright ©2006.
Chapter 4 – Reliability Observed Scores and True Scores Error
VALIDITY AND RELIABILITY
Chapter 5 Measurement, Reliability and Validity.
Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If.
Part II Sigma Freud & Descriptive Statistics
Part II Sigma Freud & Descriptive Statistics
Measurement the process by which we test hypotheses and theories. assesses traits and abilities by means other than testing obtains information by comparing.
MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.
Methods for Estimating Reliability
Measurement. Scales of Measurement Stanley S. Stevens’ Five Criteria for Four Scales Nominal Scales –1. numbers are assigned to objects according to rules.
Reliability and Validity of Research Instruments
Part II Knowing How to Assess Chapter 5 Minimizing Error p115 Review of Appl 644 – Measurement Theory – Reliability – Validity Assessment is broader term.
Reliability n Consistent n Dependable n Replicable n Stable.
A quick introduction to the analysis of questionnaire data John Richardson.
FOUNDATIONS OF NURSING RESEARCH Sixth Edition CHAPTER Copyright ©2012 by Pearson Education, Inc. All rights reserved. Foundations of Nursing Research,
Research Methods in MIS
1 Copyright © 2011 by Saunders, an imprint of Elsevier Inc. Chapter 10 Clarifying Measurement and Data Collection in Quantitative Research.
Measurement and Data Quality
Instrumentation.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 14 Measurement and Data Quality.
McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. Educational Research: Fundamentals.
Fundamental Statistics in Applied Linguistics Research Spring 2010 Weekend MA Program on Applied English Dr. Da-Fu Huang.
Reliability Chapter 3.  Every observed score is a combination of true score and error Obs. = T + E  Reliability = Classical Test Theory.
1 Chapter 4 – Reliability 1. Observed Scores and True Scores 2. Error 3. How We Deal with Sources of Error: A. Domain sampling – test items B. Time sampling.
Tests and Measurements Intersession 2006.
Reliability & Agreement DeShon Internal Consistency Reliability Parallel forms reliability Parallel forms reliability Split-Half reliability Split-Half.
Independent vs Dependent Variables PRESUMED CAUSE REFERRED TO AS INDEPENDENT VARIABLE (SMOKING). PRESUMED EFFECT IS DEPENDENT VARIABLE (LUNG CANCER). SEEK.
Issues in Validity and Reliability Conducting Educational Research Chapter 4 Presented by: Vanessa Colón.
CHAPTER OVERVIEW The Measurement Process Levels of Measurement Reliability and Validity: Why They Are Very, Very Important A Conceptual Definition of Reliability.
Copyright © 2008 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Assessing Measurement Quality in Quantitative Studies.
Reliability n Consistent n Dependable n Replicable n Stable.
©2011 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
DENT 514: Research Methods
Measurement Experiment - effect of IV on DV. Independent Variable (2 or more levels) MANIPULATED a) situational - features in the environment b) task.
Chapter 6 - Standardized Measurement and Assessment
Reliability a measure is reliable if it gives the same information every time it is used. reliability is assessed by a number – typically a correlation.
Reliability When a Measurement Procedure yields consistent scores when the phenomenon being measured is not changing. Degree to which scores are free of.
Chapter 6 Norm-Referenced Reliability and Validity.
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 11 Measurement and Data Quality.
5. Evaluation of measuring tools: reliability Psychometrics. 2011/12. Group A (English)
© 2009 Pearson Prentice Hall, Salkind. Chapter 5 Measurement, Reliability and Validity.
Data Collection Methods NURS 306, Nursing Research Lisa Broughton, MSN, RN, CCRN.
ESTABLISHING RELIABILITY AND VALIDITY OF RESEARCH TOOLS Prof. HCL Rawat Principal UCON,BFUHS Faridkot.
Ch. 5 Measurement Concepts.
Lecture 5 Validity and Reliability
Product Reliability Measuring
Assessment Theory and Models Part II
RELIABILITY OF QUANTITATIVE & QUALITATIVE RESEARCH TOOLS
CHAPTER 5 MEASUREMENT CONCEPTS © 2007 The McGraw-Hill Companies, Inc.
Journalism 614: Reliability and Validity
Reliability & Validity
Introduction to Measurement
Instrumentation: Validity Measuring Caring in Nursing
First study published in JOGS.
PSY 614 Instructor: Emily Bullock, Ph.D.
Evaluation of measuring tools: reliability
MANA 5341 Dr. George Benson Measurement MANA 5341 Dr. George Benson 1.
The first test of validity
Measurement Concepts and scale evaluation
Procedures for Estimating Reliability
Chapter 3: How Standardized Test….
Presentation transcript:

Instrumentation: Reliability Measuring Caring in Nursing

Reliability: Quality Measure of Instrument Dependability, stability, consistency, predictability, comparability, accuracy Indicates extent of random error in measurement method Test is reliable if observed scores are highly correlated with its true scores How consistently measurement technique measures concept/construct (same trait) (internal consistency) If two data collectors observe same event and record observations on instrument, recording is comparable (interrater) Same questionnaire administered to same individuals at two different times, individuals’ responses remain same (test-retest)

Reliability Instruments that are reliable provide values with only small amount of random error Reliable instruments enhance the power of the study 1.00 is perfect reliability; 0.00 is no reliability

Types of Reliability: Test-Retest test-retest (two administrations of same test with time period between administrations based on construct) comparison of means t-test: t statistic OR correlate results Pearson Product Moment Correlation or r or Spearman’s rho Time 1: administer the CBI to a group of nurses Time 2: administer the CBI to the same group of nurses Test-retest reliability, nurse sample (rho = .88, p < .01)

Types of Reliability: Equivalence Interrater or interobserver: 2 observers independently observe behavior using same instrument percentage of agreement: number of agreements/number of possible agreements = interrater reliability Cohen’s kappa, correlation (Phi, Pearson r; Kendall’s tau; Spearman’s rho) intraclass correlation coefficient (ANOVA) Alternate or parallel forms (two parallel tests based on same blueprint) Pearson r on total scores

Types of Reliability: Homogeneity Internal consistency Cronbach’s alpha (ordinal scale items) Modest sample size: at least 25 Kuder-Richardson 20 (nominal, dichotomous items)

Reliability In a study of antepartum and postpartum patients, reliability coefficients for the Caring Behavior Assessment generated an internal consistency Cronbach’s alpha of .93 for the total scale Cronbach’s alpha coefficients were the following for CBI-42 subscales: Respectful deference to other (α = .94); Assurance of human presence (α = .96); Positive connectedness (α = .93); Professional knowledge and skill (α = .90); Attentiveness to the other’s experience (α = .91) Reliability for the CARE-Q was initially tested in a small (N = 10) test-retest study, resulting in an r of 1.00 for one most important and one least important item The revised 10-item Caring Factor Survey (CFS), D model: a Cronbach’s alpha of .96 was the final version of the 10-item CFS

Reliability Five bilingual nurses in the United States examined the consistency between the modified Chinese version of the CARE-Q and its original English version. The modified Chinese version was rated as 96% accurate for the 50 items. The internal consistency for an overall coefficient alpha is 0.97 for the total scale of the modified CARE-Q (Chinese version). The test-retest reliability of the instrument with a 2-week interval was p = 0.005. CBI-42 has demonstrated good test-retest reliability (r = 0.82) for nurses The Cronbach’s alpha was 0.94 for the total CBI-24 scale (Ireland and United State nurse sample)

Reliability The Adapted Caring Behavior Checklist was used in a scenario-based, high fidelity human patient simulation based on McDaniel’s checklist; items were scored on verbal and nonverbal actions denoting caring as performed by the nurse and as observable to a rater, using dichotomous (presence or absence) scoring and frequencies. Inter-rater reliability among four raters was 88% to 100% for the adapted checklist. The Peer Group Caring Interaction Scale (PGIS) was adapted for staff nurses; the alpha reliability of the PGCIS was 0.95. The Caring Nurse Observation Tool (CNOT) checklist was tested for inter-rater reliability by two pairs of independent judges; there was a 38% and 41% agreement by two pairs of independent judges; there was a 38% and 41% rate of agreement among two independent judges within each dataset; Kappa coefficients were also calculated. An observed indicator receives a score of 1 for each item.

Caring Behaviors Inventory-42 Respectful deference to other (courteous regard for the other) (3,8,1,9,11,2,4,10,7,27,15,28) 12 items; Cronbach’s alpha = 0.8906 Assurance of human presence (investment in the other’s need and security) (16,35,37,34,32,36,30,26,33,31,18,29) 12 items; Cronbach’s alpha = 0.9221 Positive connectedness (optimistic and constant readiness of the part of the nurse to help the other) (12,25,5,14,13,6,23,21,17) 9 items; Cronbach’s alpha = 0.8452  Professional knowledge and skill (proficient, informed and skillful nurse) (19,24,22,38,20) 5 items; Cronbach’s alpha = 0.8157  Attentiveness to other’s experience (appreciation of and engrossment in the other’s perspective and experience) (40,39,41,42) 4 items; Cronbach’s alpha = 0.8191

Modified (Chinese) Care Q Subscales Accessible: α = 0.82 Explains and Facilitates: α = 0.86 Anticipates and Comforts: α = 0.92 Respects: α = 0.92 Helping and Trusting Relationships: α = 0.89 Monitors and Follows Through: α = 0.91

Persian Caring Dimension Inventory-25 PCDI re‑administered 10 days after the first visit to 20 nursing students and 18 nurses to evaluate test-retest reliability. Test–retest correlation coefficients were high (Spearman correlation coefficients were 0.89 for nursing students and 0.91 for nurses). Internal consistency was calculated for entire sample of 288 participants. Alpha coefficient for overall CDI‑25 was 0.91. By excluding two items, items 4 (getting to know the patient as a person) and 16 (sharing your personal problems with a patient), alpha coefficient for remaining 23 items was 0.86

References Cossette & Forbes (2012) DiNapoli, Nelson, Turkel, & Watson (2010) Dunnington & Farmer (2015) Larson (1984) Lee & Larson (2006) Longo (2009) Salimi, Azimpour, Mohammadzadeh, & Fesharaki (2014) Schultz, Bridgham, Smith, & Higgins (1998) Weathers et al. (2015) Wolf et al. (1994) Wolf et al. (1998) Wolf et al. (2003) Wolf et al. (2004)