1. Data Collection, Measurement, &
Data Quality in Quantitative and Qualitative Research

2. Data Collection Methods
Without appropriate data collection methods, the validity of research conclusions is easily challenged

3. Data Collection Methods
Using New Data
Collect own data for the study

4. Data Collection Methods
Using Existing Data
Historical research
Use records and other documents from the past
Secondary analysis
Use of data gathered in a previous study

5. Key Dimensions of Data Collection Methods
Structure
The data collection should be very structured and consistent
Quantifiability
Able to be analyzed statistically
Obtrusiveness
Degree to which people are aware that they are being studied
Objectivity
Try to be as objective as possible

6. Quantitative Research
Data Collection
Quantitative Research

7. Types of Data Collection
Self-Reports
Observation
Biophysiologic Measures

8. Types of Data Collection
Self-Reports
Interviews
Questionnaires
Scales
Vignettes
Projective techniques
Q-sorts

9. Types of Data Collection: Self-Reports
Interviews and Questionnaires (Structured)
Participant's responses to questions by researcher
Data is usually collected by means of a formal, written document (instrument)
Uses an interview schedule for questions that are asked orally (face to face or via phone)
Uses a questionnaire when participants complete the instrument themselves

10. Types of Data Collection: Self-Reports
Interviews and Questionnaires (Structured)
Closed-ended questions (fixed alternative questions)
Response alternatives are specified by the researcher
Ensures comparability of responses
Facilitates analysis
Easy to administer
More efficient time use
Difficult to develop
Could lead to overlooking something important

11. Types of Data Collection: Self-Reports
Interviews and Questionnaires (Structured)
Open-ended questions
Allows participants to respond to questions in their own words
Allows for richer, fuller information

12. Types of Data Collection: Self-Reports
Interviews and Questionnaires (Structured)
Instrument Construction
Develop outline of content of research
Design questions
Pretest
Trial run to determine if instrument is free of biases, errors, etc

13. Types of Data Collection: Self-Reports
Interviews Vs. Questionnaires
Advantages of questionnaires
Less costly
Require less time and effort to administer
Can be completely anonymous
No biases relating to the researcher being present

14. Types of Data Collection: Self-Reports
Interviews Vs. Questionnaires
Advantages of Interviews
Response rate is higher in face to face interviews
Effective for those that can not complete questionnaires (children, blind, ESL, elderly)
Questions are less likely to be misinterpreted than questionnaires
Interviews can produce additional information through observation

15. Types of Data Collection: Self-Reports
Interviews Vs. Questionnaires
Interviews are considered to be superior to questionnaires

16. Types of Data Collection: Self-Reports
Types of Self-Reports (Structured)
Composite Scales (social - psychological)
Vignettes
Projective techniques
Q sorts

17. Types of Data Collection: Self-Reports
Composite Scales (social - psychological)
Scale: assigns a numeric score to people to place them on a continuum with respect to attributes being measured

18. Types of Data Collection: Self-Reports
Composite Scales (social - psychological)
Likert scale
Semantic Differential scale
Visual Analog scale

19. Types of Data Collection: Self-Reports
Composite Scales (social - psychological)
Likert scale (summated rating scales)
Consists of several declarative statements that express a viewpoint
Participant indicates the degree to which they agree to disagree
Able to summate the scores allowing for discrimination among people with different viewpoints

20. Types of Data Collection: Self-Reports
Composite Scales (social - psychological)
Example Likert Scale:
AU nursing students are very well prepared for working within the current healthcare system
Strongly agree Agree Neutral Disagree Strongly disagree

21. Types of Data Collection: Self-Reports
Composite Scales (social - psychological)
Semantic Differential
Participants rate a concept on a series of bipolar adjectives
Can measure any concept
Visual Analog Scale
The scale is a straight line with anchors which are the extreme limits of the experience or feeling
Measures subjective experiences

22. Types of Data Collection: Self-Reports
Semantic Differential
Example
AU nursing graduates are:
Competent Incompetent
Intelligent Dim
Visual Analog Scale
On a scale of 0 to 10 how would you rate your pain if 10 was the worst pain you have even experienced and 0 was no pain

23. Advantages of Scales
Scales allow researchers to efficiently quantify the strength and intensities of individual characteristics
Discriminates among people with different attitudes, fears, motives, perceptions, personality traits, needs
Good for group and individual comparisons
Can be implemented either verbally or in writing

24. Disadvantages of Scales
Response set biases
Social Desirability Response Set Bias
Participants give answers that are common social views
Extreme Response Set Bias
Participants express attitudes or feelings in the extreme (always, never)
Acquiescence Response Set Bias
Participants agree with all statements (yea-sayers or nay-sayers)

25. Disadvantages of Scales
Ways to Reduce Response Set Biases
Counterbalancing: positively and negatively worded statements
Developing sensitively worded questions
Creating a permissive, nonjudgmental atmosphere
Guaranteeing confidentiality

26. Types of Data Collection: Self-Reports
Vignettes
Brief description of events or situations to which participants are asked to react
Information about perceptions, opinions, or knowledge
Questions post vignettes may be open-ended or close-ended
Economical to administer
May contain response biases

27. Types of Data Collection: Self-Reports
Projective Techniques
Verbal self reports to obtain psychological measurements
Seek minimal participants’ conscious cooperation
Ambiguous or unstructured stimuli elicits participants needs, motives, attitudes, personality traits
i.e. Inkblot test, word association, role playing, drawing
Useful in children, hearing or speech impaired

28. Types of Data Collection: Self-Reports
Q Sorts
Uses a set of card with words, phrases or statements
Participant sorts cards along a bipolar dimension (agree/disagree)

29. Advantages of Self-Reporting Methods
Most common method of data collection used by nurses
Reveal information that is difficult to obtain by other means
Can gather retrospective and prospective data
Can measure psychological characteristics

30. Disadvantages of Self-Reporting Methods
Questionable validity and accuracy
Biases

31. Types of Data Collection: Observation
Observational Methods
An alternative to self-reports
Can be used to gather information such as characteristics, condition of individuals, verbal communication, nonverbal communication, activities, environmental conditions

32. Types of Data Collection: Observation
Observational Methods
Researcher has flexibility in the following areas:
The focus of observation
What events are to be observed
Concealment
Duration of observation
Method of recording observations

33. Types of Data Collection: Observation
Observational Methods (structured)
Categories and checklists
Rating Scales

34. Types of Data Collection: Observation
Categories and Checklists
Category system:
attempts to designate information in a systematic, quantitative manner
Clear definition of behaviors and characteristics to be observed is necessary
Lists all behaviors or activities the observer wants to observe and records occurrences
Checklist:
instrument to record observations
Rating Scales:
Are tools that require the observer to rate some phenomena along a descriptive continuum

35. Types of Data Collection: Observation
Observational Sampling
Time sampling
Selection of time periods for observations
Event sampling
Selects behaviors or events for observation

36. Evaluation of Observational Methods
Advantages
Provides depth and variety of information
Some problems are better suited to observation
Disadvantages
Potential ethical issues
Lack of consent to be observed
Participants reaction to be observed
Biases
Faulty inferences

37. Types of Data Collection
Biophysiologic

38. Types of Data Collection: Biophysiologic
Types of Biophysiologic Measures
In vivo
Measures performed directly within or on living organisms
i.e. blood pressure, temperature
In vitro
Data gathered from participants by extracting some biophysiologic material from them for lab analysis
i.e. blood work, microbiologic measures, cytology and histological measures

39. Advantages of Biophysiologic Measures
Are relatively accurate and precise
Are objective
Provide valid measures of targeted variables
Equipment is readily available

40. Disadvantages of Biophysiologic Measures
Measuring tool may affect variables it is attempting to measure
Interferences may create artifact
Energy must often be applied to the organism when taking measurements

41. Measurement and Assessment of Data

42. Measurement Involves rules for assigning numeric values to qualities
Determines how much of an attribute is present
Quantification
Communicates the amount in numbers

43. Advantages of Measurement
Removes guesswork in gathering information
Tends to be objective
Obtains precise information
Can differentiate among people who possess different degrees of an attribute
Common language

44. Errors of Measurement Always the potential for error in all tools
Extraneous factors affect measurement and distort results
Obtained score – is observed score
True score – true score if no errors
Error of measurement – the different between the true and obtained scores

45. Factors Contributing to Errors of Measurement
Situational contaminants
People’s awareness of observer, environmental factors
Response set biases
Transitory personal factors
Fatigue, mood, hunger (temporary)
Administration variations
Alterations in data collection methods
Item sampling
Errors introduced as a result of sampling

46. Reliability of Measuring Instruments
Refers to the consistency with which an instrument measures the attribute
The less variation in repeat measures the higher its reliability

47. Reliability of Measuring Instruments
Aspects of reliability
Stability
Internal consistency
Equivalence

48. Reliability of Measuring Instruments
Stability
The extent to which the same scores are obtained when the instrument is used with the same people on separate occasions
To assess stability: Test-retest reliability
researcher administers the same measure to a sample of people on two occasions and then compares the scores

49. Reliability of Measuring Instruments
Internal Consistency
Reliable to the extent that all its subparts measure the same characteristic
To assess internal consistency: Split-half technique
the items comprising the test or scale are split into two groups and scored, compute reliability coefficient

50. Reliability of Measuring Instruments
Equivalence
Determines the consistency or equivalence of the instrument by different observers or raters
To assess equivalence – interrater (interobserver) reliability
Has two or more trained observers make simultaneous, independent observations, compete reliability coefficient

51. Reliability of Measuring Instruments
Reliability Coefficients
A quantitative statistic that estimates how reliable an instrument is
Determine an instrument’s quality
Low reliability makes it difficult to adequately test research hypothesis
If sample too homogeneous, the lower reliability coefficient will be (instruments are designed to measure differences)

52. Validity of Measuring Instruments
Is the concern whether the measurement tools actually measure what they are supposed to measure

53. Validity of Measuring Instruments
Aspects of Validity
Face validity
Content validity
Criterion-related validity
Construct validity

54. Validity of Measuring Instruments
Face validity
Whether the instrument looks as though it is measuring the appropriate construct

55. Validity of Measuring Instruments
Content Validity
Concerned with adequacy of coverage of the content area being measured
Tests of knowledge
Psychosocial traits
Based on judgment

56. Validity of Measuring Instruments
Criterion-Related Validity
Wants to establish the relationship between scores on an instrument and some external criterion
Compute a validity coefficient – correlates scores on the instrument with scores in the criterion variable

57. Validity of Measuring Instruments
Construct Validity
Concerned with what construct is the instrument actually measuring

58. Validity of Measuring Instruments
To assess construct validity–
known-groups technique
Groups that are expected to differ on certain attributes are administered the instrument then scores are compared
Factor analysis
Statistical procedure
Examination of relationships based on theoretical predictions

59. Reliability of Measuring Instruments
If a measuring device is not reliable, it can not be valid
High reliability of an instrument provides no evidence of its validity
Low reliability is evidence of low validity
An instrument can be reliable without being valid
Reliability consistently measures accurately
Validity measures what it is supposed to

60. Data Collection
Qualitative Research

61. Questions for Thought
What are the systematic rules for analyzing qualitative data?

62. Qualitative Data Collection Types of Self-Reports - Unstructured
Self-Reports Methods (Unstructured)
Interviews
Diaries
Observation

63. Qualitative Data Collection Types of Self-Reports - Unstructured
Interviews
Flexible
Not directed by set questions
Interviews are conversational in nature
Usually interviews are long
Can be tape recorded or researcher may take notes

64. Qualitative Data Collection Types of Self-Reports - Unstructured
Completely unstructured interviews
Start with broad (grand tour) questions
Further questions are guided by initial responses – one question's answer leads to the next question
Focused or semi-structured interviews
Researcher lists topics that must be covered in an interview
Uses a topic guide to ensure all question areas are covered

65. Qualitative Data Collection Types of Self-Reports - Unstructured
Focus group interviews
Interviews with groups of 5 to 15 people whose opinions and experiences are solicited simultaneously
Uses topic guide to guide questions

66. Qualitative Data Collection Types of Self-Reports - Unstructured
Life Histories
Narrative self-disclosures about life experiences
Has informants describe experiences in chronological order
Orally or written

67. Qualitative Data Collection Types of Self-Reports - Unstructured
Diaries
Have informants maintain daily logs of some aspect of their lives

68. Qualitative Data Collection: Observational Methods - Unstructured
Unstructured observation
Attempt to see the world as the participants see it
Participant observation – data collector actually participates in the group
Participation can be from the role as an observer or totally immersed in the social setting as a participant
Researcher needs to gain entrée into the social group under investigation
Researcher needs to establish rapport and develop trust within the group

69. Qualitative Data Collection: Observational Methods - Unstructured
Observational Data Collection
Physical setting
In what context is the human behaviour occurring
Participants
Information about the participants, what are their roles, characteristics
Activities
What are the participants doing
Frequency and duration
Specific information about the activity

70. Qualitative Data Collection: Observational Methods - Unstructured
Process
How is event occurring
Outcomes
Why is the activity occurring and what are the results
Single positioning
Staying in a single location
Multiple positioning
Involves moving around to observe behaviour from different perspectives
Mobile positioning
Involves following a person throughout a given activity

71. Qualitative Data Collection: Observational Methods - Unstructured
Observational Data Recording
Uses logs and field notes
Log – records daily events
Field notes – observer’s efforts to record information and understand data
Observational notes – descriptions of events and conversations
Theoretical notes – interpretive attempts to attach meaning to observations
Methodologic notes – instructions about what observations that need to be made
Personal notes – comments about researcher’s own feelings

72. Assessment of Qualitative Data
Do the measures used by the researcher yield data reflecting the truth
Qualitative research attempts to do this through establishing the data’s trustworthiness

73. Assessment of Qualitative Data
Establish Trustworthiness by assessing:
1. Credibility
2. Dependability
3. Confirmability
4. Transferability

74. Assessment of Qualitative Data: Trustworthiness
1. Credibility
Confidence in the truth of the data
Prolonged engagement and persistent observation
Sufficient time to collect data, focus on the phenomena being studied

75. Assessment of Qualitative Data: Trustworthiness
Triangulation
Use of multiple referents to draw conclusions, attempts to distinguish true information from errors
Data Source Triangulation
Multiple data sources (interviewing diverse informants on same topic)
Investigator Triangulation
Using more than one person to collect data
Theory Triangulation
Using multiple perspectives to interpret data
Method Triangulation
Using multiple methods (observation and interviews)

76. Assessment of Qualitative Data: Trustworthiness
External checks: Peer debriefing and member checks
Peer debriefing – review and explore various aspects of inquiry with objective peers
Member checks – providing feedback to study participants and assessing their reactions
Searching for Disconfirming evidence
Search for data that challenges the emerging conceptualization or theory
Researcher credibility

77. Assessment of Qualitative Data: Trustworthiness
2. Dependability
Data stability over time and over conditions
Stepwise replication
Having several researchers break into teams and evaluate the data separately and then compare conclusions
Inquiry audit
Scrutiny of the data and supporting documents by an external reviewer

78. Assessment of Qualitative Data: Trustworthiness
3. Confirmability
The objectivity or neutrality of the data, can other independent people agree about data’s relevance
Audit trail – documentation that allows an independent auditor to come to the same conclusions about the data

79. Assessment of Qualitative Data: Trustworthiness
4. Transferability
The extent to which the findings from the data can be transferred to other settings or groups

80. Reference
Loiselle, C. G., Profetto-McGrath, J., Polit, D. F., & Beck, C. T. (2011). Canadian essentials of nursing research. (Third Edition). Philadelphia: Lippincott, Williams & Wilkins.