Class 7 Jeff Driskell, MSW, PhD Research In Social Work Practice Salem State University School of Social Work Class 7 Jeff Driskell, MSW, PhD

Class Agenda Check-in/Questions Lecture- Quantitative Measurement

Measurement

Measurement Defined “Measurement is the process of observing and recording the observations that are collected as part of a research effort”.

Quantitative vs Qualitative Measurement Deductive- begins with a concept then creates measures to capture in a numerical form Reflection on concepts before they gather data More objective Qualitative Inductive approach measuring features of social life as part of the process that allows for the creation of new concepts or theories. Simultaneously reflects on data during the data collection process. More subjective

Why is Measurement Important?

Quantitative Research Moving from problem formulation to being clear about what you specifically want to study as well as how you define and measure your concepts. This process is known as conceptualization and operationalization

Shaping and Refining Concepts Conceptualization = Operationalization = A process of developing a definition of concepts (variables) Identifying how you will actually measure the concepts /constructs (variables) under study. Observable terms

The Goal….. To devise operations that measure the concepts we intend to measure—in other words to achieve measurement validity.

Measurement Requires 3 Things A construct/concept- Example- Morale A measure- Survey or scale Observation of teachers An ability to know what you are looking for Distinguish morale from other things (clearly defined)

The Multi-Dimensions of Conceptual Definitions What is a Table ? What is a Car ?

In a Nutshell… Moving from concepts to measureable observations It’s about operationalization Moving from theory to reality It’s about objectivity Working to objectively measure concepts Does everyone agree on “know it when you see it?” As objective as possible vs. subjective personal judgment  Importance of consistency with the literature (i.e. what other people do)

Measurement Error Measurement Error: Data do not accurately portray the concept we attempt to measure Systematic error Random error

Systematic Error When the information we collect consistently reflects a false picture Biases: The most common way our measures systematically measure something other than what we think they do is when biases are involved, e.g.: Acquiescent response set Social desirability bias

Random Error Random errors have no consistent pattern of effects. Examples: Cumbersome, complex, boring measurement procedures Measure uses professional jargon which respondents are not familiar with

Avoiding Measurement Error Use unbiased wording Carefully train interviewers Understand how existing records are kept Triangulation Using several different research methods to collect the same information

Measurement in Action

Concrete Vs Abstract Concepts

Conceptualization Example The DSM defines substance abuse as the “repeated use of a substance to the extent that it interferes with adequate social, vocational, or self-care functioning.” Many researchers use the above definition Definition is stated in clear and precise language It also reflects a particular theoretical orientation (the “disease model”)

Operationalize- Indicators Substance Abuse/dependency

Operationalization Example MAST (Michigan Alcoholism Screening Test) 24-item instrument (yes/no) Includes a variety of indicators (drinking as a problem, seeking treatment, delirium tremens, severe shaking, memory loss, social problems, arrests).

Operationalization Example: CAGE Screener "Have you ever felt you should Cut down on drinking (drug use)?" "Have people Annoyed you by criticizing your drinking (drug use)?" "Have you ever felt Guilty about your drinking (drug use)?" "Have you ever taken a drink (used drugs) in the morning to steady your nerves or get rid of a hangover (“Eye opener”)? Scoring: Answering YES to 2 questions provides strong indication for substance abuse or dependency Answering YES to 3 questions confirms the likelihood of substance abuse or dependency

Conceptualization in Action: Depression In small groups, discuss your conceptual definition of depression. Locate 2-3 articles (keep them for the follow up exercise) on depression. How is depression conceptualized? Do the authors define the concepts you identified? If so, what is the definition? What are the indicators/dimensions of the concepts? How is it measured?

Reliability and Validity in Measurement

RELIABILITY vs VALIDITY Evaluating Measures RELIABILITY vs VALIDITY

Reliability The consistency of a measurement procedure. The “dependability “factor. Determining reliability Test-re-test- Measures the score twice with the same instrument. Reliable measures should produce very similar scores.

Reliability cont… Inter-rater/observer reliability- (qualitative) When observers must use their own judgment to make interpretations. Ie. Observing video-taped behaviors, interpreting open-ended answers on a questionnaire. Strategy- have different observers take measurements of the same responses- the agreement or disagreement between raters is known as inter-rater reliability. Can be compared statistically.

A test of reliability Chronbach’s alpha: α Often reported as internal consistency Rule of thumb: Alpha score of .6-.7 is acceptable Alpha score of .8 or higher is good

Example- Reliability factor

Ways to Improve Reliability Clearly conceptualize all constructs Increase the level of measurement Use multiple indicators of a variable Use pre-tests or pilot studies

Example- Substance Abuse Three things to ask when reviewing literature on substance abuse: What is meant by substance abuse in the research literature (conceptualization)? How is substance abuse measured (operationalized)? Is the measurement method accurate and valid (reliability and validity).

Validity Types of Validity Construct Face Content The degree in which the measurement tool measures what it is supposed to measure and does so accurately. Types of Validity Construct Face Content

Exercise Based on the readings assigned for today answer the following: What is the DV? What is the name of the measurement tool used to measure the identified concept? Identify two scales used in each of the tow articles. What is said about the internal consistency of the measurement tool/Cronbach’s alpha?

Use of Scaling Questions Most common- Likert Scale The final score is the sum of their ratings for all of the items- known as "summated" scale. Example- Response and scoring 1 = strongly disagree 2= disagree 3= undecided 4= agree 5= strongly agree

Levels of Measurement

Levels of Measurement Nominal Ordinal Interval Ratio

Activity- Levels in Action In pairs- Define each level of measurement Provide an example of each level of measurement, i.e. gender Last- create a total of 4 survey questions using each of the 4 levels of measurement.

Levels of Measurement A given variable can sometimes be measured at different levels of measurement (e.g., age)