Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved in converting a design into a study to run

Conceptualisation We want to speak of abstract things “Intelligence”, “ability to cope”, “life satisfaction” We cannot research these things until we know exactly what they are Conceptualisation os the process of defining terms before research Once a thing has been conceptualised it is a “Construct”

Making conceptual definitions Begin with the lay understanding of the definition This will be understood by the subjects Then consult the experts (literature) Can be confusing, contradictory Create a preliminary definition “Test it” hypothetically Use thought experiments

The danger of reification You must not try to make constructs out of things that don’t exist – reification Careful grounding of the construct in extablished theory will prevent this Eg. Is Homophobia a construct? (is it not just prejudice)? Using reified constructs leads to empty, disconnected research

Operationalising variables Your design specifies the variables How do you measure the variables? How do you put a number to “intelligence”? How do you put a number to “capacity to cope”? We need to convert abstract variables into things which we can measure in the real world - operationalisation

Operationalisation (2) Turn your variable into a directly measureable thing Eg: How would you operationalise “success at university?” Often there are developed scales available If you operationalise badly, you end up not studying what you want Eg. Operationalising “success in career” by looking at the paycheque only

Measuring variables The operationalisation implies what to measure variable – how do you do it? If at all possible, use an established scale If no scale exists, construct one Scales must be valid and reliable The more of each of these properties, the better the scale Validity and reliability need to be sorted out before you run your study

Reliability in scales Reliability: stability of a measure over time If I measure you now and then in half an hour, do I get the same reading? Max reliability depends on the construct Some construct are unstable (eg. heart rate) Low reliability implies that other variables (“noise variables”) are being measured also Speaks of the “accuracy” of the scale

Ensuring reliability Reliability suffers when subjects have to interpret Everyone’s interpretation is slightly different Objective scales are always more reliable Allow little interpretation Using a fixed response format helps Eg. Multiple choice, Likert type Researcher does not have to interpret what the subject meant

Examples of response types Open ended item: Briefly describe your most frightening experience MCQ: The most frightening for me is A) Dogs B) Snakes C) Spiders D) None of the above

More examples Likert type: Circle the one which best describes your experience. I find dogs to be Not frightening at all Terrifyingly frightening

Validity in scales Validity: the degree to which a scales measures what it is supposed to Validity is subdivided into many types We will look at 2 most important Criterion Realted Validity Construct Validity

Criterion Related Validity The degree to which this scales matches other established scales By comparing to a scale known to be valid, you can be sure yours is valid Why make a new scale if one already exists? Maybe yours is quicker to do Maybe the established is not for group testing

How to check for criterion related validity This is done through a set of studies Run a sub-study in which you give the subjects your scale and the established one Run a correlation between the two scales If the correlation is statistically significant, your scale compares well to the established one. It is better to run several of these validity studies rather than just one.

Example: intelligence test An accepted test is the WAIS-R Very long to run (3 hours) You need something quicker (20 minutes), create the QIQ Create a test, select a group of subjects Make them take the WAIS-R and then the QIQ Compare the results (correlation) If they correlate well, your test is measureing intelligence

Construct Validity Construct validity: Does the scale actually measure the construct? Eg: measuring cranial circumference to measure intelligence Closely tied into the theory of the construct Most difficult to achieve, most important Measures lacking in construct validity are almost useless

How to check for construct validity Think abou it for a minute: How can you show that a scale truly measures what it claims to? How would you show that your depression scale has construct validity? Hint: Compare it not to scales of the same thing, but to similar and dissimilar things

The strategy Similar procedure to criterion related validity: Before your actual study, run a set of sub- studies to check your measure You will need 2 sets of studies Concurrent construct Validity Discriminant construct validity

Quick aside: direction of correlations Correlation: the degree of relationship between two variables, A and B Positive correlation: when A has a high value, B has a high value. When A has a low value, B has a low value Negative correlation: when A has a high value, B has a low value. When A has a low value, B has a high value

Correlations example Positive correlation: the relations ship between amount smoked and probability of heart disease Negative correlation: the relationship between amount of daily exercise and probability of heart disease No correlation: the relationship between whether you drink tea or coffee and the probability of heart disease

Concurrent validity Show that your scale relates positively to related concepts People who do are depressed will have many sad thoughts (mood conguency effect) Establish concurrent validity against several other constructs

Discriminant validity Show that your scale relates negatively to opposite concepts People who are depressed will have very low energy Establish discriminant validity against several other constructs

Ensuring construct validity Best way: be an expert on that construct Theory should tell you what things to include BUT: only if the theory is well-established! Second way: consult the experts/literature closely Stay with the uncontroversial aspects of that construct

Validity & reliability summary Aim: make sure that your variables are correctly operationalised Reliability: scale is stable over time/place Validity: scale is truly measuring the construct not something else

Validity & Reliability summary (2) Ensuring reliability: require verly little interpretation / increase objectivity Ensuring validity: base the measure closely on current understanding of construct Measuring validity: positive correlations with related scales, negative correlations with opposite scales