Research Methods for Counselors COUN 597 University of Saint Joseph Class # 3 Copyright © 2015 by R. Halstead. All rights reserved.

Class Objectives  Basic Overview of Measurement  Trochim Chapter 3  Overview of Survey Research  Trochim Chapter 4  Review of Correlation Coefficients  Salkind Chapter 5  Some Practice with Concepts

Topics Appropriate for Survey Research  There are a variety of uses for surveys  Individuals are the unit of analysis  Descriptive - (U.S. Census)  Explanatory - (Attitudes individuals hold)  Exploratory - (Discover some new aspect or characteristic dimension such as in a needs assessment)

Self-Administered Questionnaires  Mail Distribution and Return  Electronic Surveys  Monitoring Returns  Follow-up Mailings  Response Rates

Interview Surveys  The Role of the Survey Interviewer  General Rules for Survey Interviewing  Appearance and Demeanor  Familiarity with Questionnaire  Following Question Wording Exactly  Accurate Recording of Responses  Probing for Responses  Coordination and Control

Telephone Surveys  Sampling Problems  Phone ownership used to be a problem now it is cell phones and the ability to access to the numbers which are unlisted  Over use by telemarketing and political parties  Advantages  Random Digit Dialing  Cost Effective - Computer Controlled Multi- Number Dialing

Strengths and Weaknesses of Survey Research  Strengths  Useful in describing the characteristics in large populations  Allows for obtaining very large sample sizes  Standardization of the questionnaire allows for greater control over factors that may serve to bias informant responses

Strengths and Weaknesses of Survey Research  Weaknesses  Standardization of the questionnaire may limit the uniqueness of informant responses  Survey research rarely allows for an understanding of the informants’ lives in social context  There is an assumptive leap that how a person answers the survey has some bearing on how that person actually operates a situation

Steps in the Survey Research Process  Questionnaire Acquisition or Construction  Sample Selection  Survey Administration (Data Collection)  Data Analysis  Drawing Conclusions

Secondary Analysis  One of the less expensive ways to engage in survey research is to conduct an analysis on research data that has been collected for another purpose.  Data archives allows researchers to do this  See web link in syllabus to the Murray Center  Also see the ACA Code of Ethics that addresses release of data to researchers

Conceptualization & Measurement  Conceptualization – The birthing of an idea  Operational definitions – Clearly defining terms  Measurement – Adopting a method or method(s) for specifying and collecting data that can be later used for analysis.

Conceptualization  Identify personal conceptions  Identify public constructs  Develop nominal definition  Develop operational definition (Operationalization)  Terms: Concepts, Constructs, Indicators, Dimensions

Conceptualization  Where Do Research Topics Come From?  Practical problems in the field  Literature in the field  Your own thinking  Is the Study Feasible?  Tradeoff between rigor and practicality  How long it will take  Ethical constraints  Needed cooperation  Costs

Conceptualization  Conducting the Literature Review  Review the scientific literature – What does it say?  Are there inconsistencies that warrant further study?  Do the review early in the process  The literature review can help you  See if your idea has been tried  Include all relevant constructs  Select instruments  Anticipate common problems

Operational Definitions  Points the way to how a variable will be measured  Specify observation procedures  Specify coding rules

Measurement  Levels of measurement  Measurement Error/Precision  Reliability  Validity

Levels of Measurement  Nominal Scale - Names and Categories  Example: Gender, Martial Status, Race  Ordinal - Rank ordering  Example: 1st, 2nd, 3th  Interval - Equal intervals between levels of an attribute  Example: Age expressed in whole years.  Ratio - Continuous data can assume any value between two point along a continuum.  Example: Time (2.35 seconds)

Levels of Measurement - So What?  Must be able to chose the level of measurement that will allow you to answer your question of interest. Here is an example. Male Female Asian Infant + MaleFemaleAsianInfant?Divided by 4 = ? Nominal Data

Measurement Error  Systematic Error - reflects a false picture because of some flaw in the system.  Acquiescence  Social desirability  Culture bias  Random Error - Inconsistencies inherent to any form of measurement

Avoiding Systematic Measurement Error  Select appropriate instruments for your project  Select instruments that have demonstrated reasonable validity and reliability statistics  Look for elements that might suggest bias  Conduct a small pilot study  Be certain co-researchers are up to the task  Attend environmental factors

Reliability  Defined as the degree to which assessment measures are consistent, dependable, and repeatable.  Inter-Rater Reliability  Test-Retest Reliability  Parallel Forms or Alternate Forms Reliability  Split-Half Reliability

Validity  A multidimensional concept use as a means of expressing the degree to which a certain inference drawn from a test is appropriate and meaningful.  An instrument measures what it purports to measure.

Types of Validity  Content Validity - The content of the items make sense - said to have Face Validity.  Criterion-related Validity - The test score is related to one or more outcome criteria of interest. (Concurrent and Predictive)  Construct Validity - Establishes that the instrument expresses an accurate measure of some construct.

The Problem  Concepts are not mutually exclusive.  They exist in a web of overlapping meaning.  To enhance construct validity, you must show where the construct is in its broader network of meaning.  Take a look at the next slide.

What Is the Goal? The construct Other construct: A Other construct: C Other construct: B Other construct: D Measure the construct and nothing else. Trochim, 2001

Example: You Want to Measure Self-Esteem Self - esteem Trochim, 2001

Example: How Would You Distinguish Self-esteem From... Self- esteem Self-worth Confidence Positive Self- disclosure Openness Trochim, 2001

To Establish Construct Validity  You have to set the construct within a semantic (meaning) net.  You have to provide evidence that your data support the theoretical structure. (Constructs that should be more related, are more related and constructs that should be less related, are less related.)  Supply evidence that you control the operationalization of the construct (that your theory has some correspondence with reality). Trochim, 2001

Summary  Reliability - speaks to the is consistency of measurement. Good reliability suggests that as the measure is used repeatedly, all factors being equal, the results will show little change.  Validity - speaks to the accuracy of the measure in labeling that which it is supposed to be measuring.

Correlation – A Review  Correlation is a statistical measurement that indicates the strength of relationship between two sets of data (two variables).  In essence, a correlation coefficient tells us how two sets of data “co-relate.”  Another way to think about this is how two variables change relative to one another.

Correlation - Continued  Samples  r xy is the correlation between variable x and variable y  r height-weight is the correlation between height and weight  r SAT-GPA is the correlation between SAT and GPA

Correlation - Continued 9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 *

Correlation - Continued  Strength  Perfect Correlation  Perfect Correlation  None 0.00  Direction  Direction - Positive (+) and Negative (-)

Absolute Value and Strength  Which is the Stronger of the two correlations below? or Answer: -.70 because the absolute value |.70 | is greater than the absolute value |.50 |.

Understanding What Correlation Means - A Rough Guideline  General Rule Size of the Correlation General Interpretation.8 to Very Strong.6 to Strong.4 to Moderate.2 to Weak.0 to Very Weak or at 0 No Relationship Good method for a quick assessment

Understanding What Correlation Means - Coefficient of Determination  The Coefficient of Determination  Defined: The percentage of variance in one variable that is accounted for by the variance in the other variable.  Remember Variability? (Salkind Chapter 3)  Variability (spread or dispersion) is a measure of how different scores are from one another.  We learned that usually variability is thought of as a measure of how much each score in a distribution differs from the mean.

Understanding What Correlation Means - Coefficient of Determination  Variables that share something in common tend to correlate with one another.  Final grade in Appraisal and in Research have a moderately high correlation for many reasons.  Similar concepts  Hours of study put into the endeavor is a consistent trait  Committed to regular practice  Aptitude for subject matter  These factors and others account for the differences in students’ grades - variability

Understanding What Correlation Means - Coefficient of Determination  The more two variables share in common the more they will be related - correlate.  The two variables are said to share variability (the reasons why the final grades in Appraisal and Research tend to be similar).  Similar Concepts  Hours of Study  Regular Practice  Aptitude for Subject Matter

Understanding What Correlation Means - Coefficient of Determination  To determine exactly how much of the variance (the dispersion or spread) one variable can be accounted for by the variance (the dispersion or spread) in another variable you just simply square the correlation coefficient.  Let’s look at an example of this.

Understanding What Correlation Means - Coefficient of Determination  Previously it was stated that we could make rough estimates regarding the strength of a correlation. Size of the Correlation General Interpretation.8 to Very Strong.6 to Strong.4 to Moderate.2 to Weak.0 to Very Weak or No Relationship

Understanding What Correlation Means - Coefficient of Determination yBy computing the shared variance (the Coefficient of Determination) we can see why a r =.8 is strong as opposed to a r =.2 r =.8 squared =.64 or 64% of the variance for each variable is shared r =.2 squared =.04 or 4% of the variance of each variable is shared

Understanding What Correlation Means  I know you have been holding back on asking the one question that is burning in the forefront of your mind - so let’s tackle that now.  What about that portion of the variance that is not shared? That portion that can not be explained by a coefficient of determination?

Understanding What Correlation Means - Coefficient of Alienation  The Coefficient of Alienation is the amount of the variance in one variable not explained by the variance in the other variable.  Logic would suggest, then, that the portion of unexplained variance must be due to factors (variables) that have not, as of yet, been taken into account.

Applying Correlation in Practice  Recall our December clients’ intakes  Lets look at their levels of depression last August and arrange to data as matched pairs

Applying Correlation in Practice  December clients’ intakes.  Mean = Md = 20 Mo = 20 Range = 33 Sd =  December clients in August.  Mean = Md = 11 Mo = X Range = 22 Sd = 6.2

Applying Correlation in Practice  December clients intakes - Var X  December clients in August - Var Y.

Applying Correlation in Practice  December clients intakes - Var X.  December clients in August - Var Y r =.38 2 r =.14  Correlation  Coefficient of Determination

Applying Correlation in Practice  Correlation  Coefficient of Determination r =.38 2 r =.14  What is this results of our data analysis saying?  What research questions might these results generate?