Research methods.

Research methods

A2: what you need to cover
Reliability across all methods of investigation. Ways of assessing reliability: test-retest and interobserver; improving reliability. Types of validity across all methods of investigation: face validity, concurrent validity, ecological validity and temporal validity. Assessment of validity. Improving validity. Content analysis Features of science: objectivity and the empirical method; replicability and falsifiability; theory construction and hypothesis testing; paradigms and paradigm shifts. Reporting psychological investigations. Sections of a scientific report: abstract, introduction, method, results, discussion and referencing.

A2: what you need to cover
Inferential testing Students should demonstrate knowledge and understanding of inferential testing and be familiar with the use of inferential tests. Introduction to statistical testing; the sign test. Probability and significance: use of statistical tables and critical values in interpretation of significance; Type I and Type II errors. Factors affecting the choice of statistical test, including level of measurement and experimental design. When to use the following tests: Spearman’s rho, Pearson’s r, Wilcoxon, Mann-Whitney, related t-test, unrelated t-test and Chi-Squared test.

Today Introduction to inferential statistics Null hypothesis Level of significance Probability Type 1 and type 2 errors Carrying out statistical tests Picking the right test Writing up the report

Inferential statistics
Lesson 1

Introduction to inferential statistics Why not just use descriptive stats?
Descriptive statistics give us convenient and easily understood summaries of the data but we can’t draw any firm conclusions from them, they are just an overview. In order to draw firmer conclusions and to accept or reject hypotheses, inferential statistics are needed.

Inferential statistics are used to test hypothesis. Do groups differ on some outcome variable? Is the difference more than expected by chance Used to make generalisations from a sample to a population. Inferential statistics take into account sampling error (chance, random error)

Null hypothesis: What is a hypothesis?
Two hypothesis are formulated at the beginning of a study: The alternative hypothesis (H1) Predicts that there will be a significant difference Directional or non directional (one tailed or two tailed) The null hypothesis (HO) predicts that there will not be a significant difference

What is a null hypothesis and why do we need one?
The null hypothesis predicts that any difference between two or more sets of data will have occurred through chance alone If it is rejected then we must retain the alternative (experimental) hypothesis If the null hypothesis is rejected we say our results are statistically significant. If it is accepted we say they are NOT significant. We focus on the null hypothesis because it eliminates bias from the research by forcing the researcher to consider the view that any difference found between the two sets of data has occurred through chance alone

How to write a Null hypothesis
Aim: Does breakfast improve memory There will be no difference between the teachers score on a memory test between those who have had breakfast (cereal and yoghurt) and those who have not had any breakfast; any differences are due to chance factors. Task: recap hypothesis and write your own null hypothesis to the following scenarios

A team of psychologists was interested in studying the effects of alcohol on peoples' reaction times. Earlier research suggested that an increase in reaction time was due to the alcohol rather than peoples' expectations of alcohol. The psychologists recruited two groups of volunteers (an independent groups design) from a local university. Each participant's reaction time was measured by using a computer game. The participants were then given a drink. The first group received a drink containing a large measure of strong alcohol; the second group received an identical drink without alcohol, but with a strong alcoholic smell. Finally, all participants were required to play the computer game again to assess their reaction time. Once they had completed the task, they were then thanked for their time and allowed to leave. What is the IV? whether the participants have had an alcoholic drink or one that is not alcoholic but smells as if it is What is the DV? reaction times on a computer game Null hypothesis: There will be no difference between the university students‘ reaction times on a computer game between those who have had an alcoholic drink or one that is not alcoholic but smells as if it contains alcohol; any differences are due to chance factors.

A teacher in a small secondary school wanted to find out whether there was any truth in her idea that students who used a computer regularly for their homework achieved higher exam grades than those who did not. She decided to interview a sample of 30 students taken from across the school. She tape-recorded all the interviews. She later obtained their end of year exam grades from their reports. What is the IV? whether the participants used a computer regularly for their homework or didn’t use a computer regularly for their homework. What is the DV? Exam grade achieved Null hypothesis: There will be no difference between the exam grades achieved at the end of year between those who regularly used a computer to complete homework and those who did not regularly use a computer to complete homework; any differences are due to chance factors.

Level of significance What is statistical significance then?!
Inferential statistics is a test of significance because it is designed to assess whether we reject or retain the null hypothesis. Inferential statistical tests work by assessing the probability of our results occurring due to chance alone (rather than the IV) We use it to determine if the probability of our results being down to chance is low enough to reject the null hypothesis

P value The reason for calculating an inferential statistic is to get a p value (p = probability) Inferential statistical tests work by assessing the probability of our results occurring due to chance alone (rather than the IV) The p value determines whether or not we reject the null hypothesis. We use it to estimate whether or not we think the null hypothesis is true. The p value provides an estimate of how often we would get the obtained result by chance, if in fact the null hypothesis were true. If the p value is small, reject the null hypothesis and accept that the samples are truly different with regard to the outcome. If the p value is large, accept the null hypothesis and conclude that the treatment or the predictor variable had no effect on the outcome.

Probability and Significance
Probability, or p, is expressed as a number between 0 and 1. 0 means an event will not happen. 1 means that an event will definitely happen. The P value will always be found to be between 0 and 1 due to the way in which it is calculated.

Task Elle listens to the radio every morning before work. The weatherman said that there is a 0.05 chance that it will rain everyday for the next 100 days. However, Elle lives life on the edge and doesn’t like to use an umbrella. In the next 100 days, how many times is she likely to get wet? 5 Days in 100 In other words there is 95% chance that Elle will not get wet. Hence she doesn’t take an umbrella.

Decision rules – Levels of significance
How small is "small?“ Once we get the p value (probability) for an inferential statistic, we need to make a decision. Do we accept or reject the null hypothesis? What p value should we use as a cutoff? The one chosen is called the level of significance.

So what is level of significance?
Significance is a statistical term which indicates that the association between two (or more) variables is strong enough for us to accept the experimental hypothesis.

Levels of significance
Researchers can use significance levels of 10%, 5%, 1% (or 0.1% in very stringent conditions) - expressed as: 10%, 0.10, 1 in 10, p≤0.10. 5%, 0.05, 1 in 20, p≤0.05 1%, 0.01, 1 in 100, p≤0.01 If you use a 5% statistical significance level and this is achieved you are saying that the probability of your results being a fluke and nothing to do with your IV is less than 5% or you are 95% sure that your change in DV is because of your IV

Probability values often used in research
P<0.01 (1% attributed to chance) – used when researchers are sure that the IV will have an effect on the DV (usually when a psychologist is replicating previous research where consistent findings have been obtained). This may also be used when psychologists are carrying out research in which the results need to be more or less guaranteed (e.g. when testing the effectiveness of a new drug). P<0.05 (5% attributed to chance) – used in most pieces of research. P<0.10 (10% attributed to chance) – used mainly when research hasn’t been carried out before. It is good practice in psychology that once a piece of research has been found to be significant at a higher level of chance (e.g. 10%) the research is then repeated under a lower level of chance (5%). The lower the value of chance, the more striking the psychological results.

Lower levels of significance
Occasionally, a more stringent level of significance may be used (such as 0.01) human cost – such as drug trials – or ‘one off’ studies that could not, for practical reasons, be repeated in future. In all research, if there is a large difference between the calculated value and critical values – in the preferred direction – the researcher will check more stringent levels, as the lower the p value is, the more statistically significant the results.

We express our results in terms of the Null Hypothesis, if a result is statistically significant we can reject the null hypothesis. If the result is not statistically significant we must accept the null hypothesis. Null Hypothesis: - Any difference between the two conditions is due to chance.

Type 1 and type 2 errors: The 5% level of significance has been accepted as it represents a reasonable balance between the chances of making a type 1 or type 2 error These can occur because: Level of probability accepted is either too lenient (too high) or too stringent (too low)

Type 1 (One-Reject) Due to the fact that researchers can never be 100% certain that they have found statistical significance, it is possible (usually up to 5% possible) that the wrong hypothesis may be accepted. A type 1 error is when the null hypothesis is rejected and the alternative hypothesis is accepted when it should have been the other way round because, in reality, the null hypothesis is ‘true’. This is often referred to as an optimistic error or false positive as the researcher claims to have found a significant difference or correlation when one does not exist. We are more likely to make a Type 1 error if the significance level is too lenient (too high) e.g. 0.1 or 10% rather than 5%.

Type 2 error A type 2 error is the reverse, when the null hypothesis is accepted but it should have been the alternative hypothesis because, in reality, the alternative hypothesis is true. This is a pessimistic error or ‘false negative’. A type 2 error is more likely if the significance level is too stringent (too low) e.g or 1%, as potentially significant values may be missed. Psychologists favour the 5% level of significance as it best balances the risk of making a Type 1 or Type 2 error.

Type 1 and type 2 errors Type 1 error: Type 2 error:
Occurs when we conclude that there IS a significant difference when there is NOT We reject the null and accept the experimental This can happen if the accepted level of probability is set TOO LENIENT Significance level set at 20% Type 2 error: Occurs when we reject the experimental hypothesis and accept the null when there IS a difference We accept the null and reject the experimental This can happen if the probability level is TOO STRINGENT Significance level set at 1%

When are errors more likely?
p = 0.05 significance level is ok as long as research is not life or death e.g. medical research (5% possibility that results are due to chance) p = 0.1 significance level, more likely to make type 1 error (10% possibility that results are due to chance) p = 0.01 significance level, more likely to make type 2 error (1% possibility that results are due to chance)

Inferential Statistics
Lesson 2

Recap: Outline the difference between descriptive statistics and inferential statistics? The null hypothesis predicts that there will be a significant difference? True/false. Shorthand for the null hypothesis is Ho? True/false What are Inferential statistics? Why is it necessary to have a Null hypothesis? 6. Explain why a 0.01 level of significance (p = 0.01) may result in a Type 2 error. 7. When would a 0.01 level of significance be used? Describe one way of checking whether a Type 1 error has been made. Extension: answer the questions on page 9/10 of your RM book

Outline the difference between descriptive statistics and inferential statistics? Summarising data vs. allowing you to see whether the research hypothesis or null hypothesis is retained The null hypothesis predicts that there will be a significant difference? True/false. False Shorthand for the null hypothesis is Ho? True/false True What are Inferential statistics? Tests designed to assess whether we reject or retain the null hypothesis. Why is it necessary to have a Null? Eliminates bias. Forces researcher to accept the view that the two sets of data has occurred through chance. Means there is no other conclusions that can be made

7. When would a 0.01 level of significance be used?
6. Explain why a 0.01 level of significance (p = 0.01) may result in a Type 2 error. P=0.01 means that there is a 1% probability that a Null Hypothesis is correct. In many experiments, this is too strict, and would result in many instances where the Alternative Hypothesis is rejected incorrectly. 7. When would a 0.01 level of significance be used? P=0.01 could be preferred when findings are likely to be controversial, raise ethical dilemmas (e.g. in medical trials), or where the results are theoretically very important. The Researcher wants to be more stringent, so they only present findings with a very small chance of the Null Hypothesis being correct. 8. Describe one way of checking whether a Type 1 error has been made. Repeat the experiment and compare the findings. If the findings have not been replicated, then it is likely that a Type 1 error has occurred.

CHALLENGE: Make up an example of a Type 1 error.
A tough decision….. Decide if the following are Type 1 or Type 2 errors: A. Damien tested his Gran to find out if she is loosing her marbles. He gives her a memory test where she has to remember lists of words 10mins after seeing them. He did a stats test on the results. Damien calculated it as p=0.3. He decided that his Gran was not loosing the plot. Maybe its him. However, the next week her GP gives her a test and decides that she has early-stage Alzheimer's Disease. Answer: Type 2 B. It has been shown many times that on a certain memory test, recognition is substantially better than recall. However, the probability value for the data from your sample was .12, so you were unable to reject the null hypothesis that recall and recognition produce the same results. What type of error did you make? Answer: Type 2 CHALLENGE: Make up an example of a Type 1 error.

Recap: hypothesis Researchers begin their investigation by writing a hypothesis. This may be directional if there is previous consistent previous research, or non‐directional. These hypotheses are often referred to as an alternative hypothesis, as it is alternative to the null hypothesis. This states there is no difference or relationship between conditions. The statistical test determines which hypothesis is ‘true’ and thus whether we accept or reject the null hypothesis.

Recap: Levels of significance and probability
Statistical tests work on the basis of probability rather than certainty. All statistical tests employ a significance level – the point at which the research can claim to have discovered a significant difference or correlation within the data. In other words, the point at which the researcher can reject the null hypothesis and accept the alternative hypothesis. The usual significance level in psychology is 0.05 (or 5%). This is properly written as p≤0.05 (p means probability) HOWEVER there is still up to 5% probability that the observed effect occurred by chance – that it was a ‘fluke.’ Psychologists can never be 100% certain about a particular result as they have not tested all members of the population under all possible circumstances! For this reason, psychologists have settled upon a conventional level of probability where they are prepared to accept that results may have occurred by chance – this is the 5% level

Statistical tests: The calculated value/critical value
Once a statistical test has been calculated, the result/value is a number ‐ the calculated value (or observed value). To check for statistical significance, the calculated value must be compared with a critical value (found in a table). Our calculated value must be greater/lower (depending on the test) in order for us to reject the null hypothesis and accept the alternative hypothesis.

Criteria for statistical test
The test that researchers use rely on these criteria: Whether the researcher is testing for differences between groups (i.e. an experiment) or a correlation between two co-variables Level of measurement (nominal, ordinal, interval and ratio) In a test of difference, whether the experimental design is an independent groups, repeated measures or matched pairs

A test of difference or a correlation?
Laboratory experiments, field experiments, natural and quasi experiments are all testing for differences between groups. The researcher is trying to establish the probability that changes in the DV are caused by the experimental manipulation or naturally occurring IV. Correlational research is attempting to show how two co-variables are linked. In this context correlation can include correlational analysis as well as investigations that are looking for an association. So look for the word correlation or association in the hypothesis for a correlation.

Difference or correlation? (page 12-13)
Researching an association between time spent on homework (1/2 hour to 3 hours) and number of G.C.S.E. passes (1 to 6). _________________________ Researching whether different verbs used in a question influences the accuracy of eyewitness testimony in the form of estimated speed given. _________________________ Researching the relationship between watching violence on T.V. and violent behaviour in adolescence. _________________________ To see if people work harder when they eat breakfast before coming to school compared to not eating breakfast when coming to school. _________________________

Experimental design (page 13-14)

Levels of measurement

Why do I need to know about the levels of measurement?
most appropriate descriptive statistic to calculate which graph to use which inferential test to use Levels of measurement relate to quantitative data.

Measuring levels of measurement
Activity Nominal: 2 groups: tall on left/short on right Tall table: short table What is problem? Issues? Form a line in the class: tallest to shortest Rank: name (table) Better – what are the problems? Lets take your actual height (shoe size) – pot a correlation Equal difference

Levels of measurement Nominal (category) data
This is the most simplest method of classifying information. Involves counting frequency data We must only be able to place each item/person into one category Only classifies each person as ‘tall’ or ‘short’; no distinction at all between ‘tall’ people.

Levels of measurement Ordinal data: This level of measurement involves ranking data into place order, with rating scales often being used to achieve this. These intervals cannot be considered equal (based on subjective opinion) They do not tell us about distances between positions We convert raw scores to ranks for statistical testing (1st ,2nd,3rd) and it is the ranks and not the scores that are used in the calculations.

Levels of measurement Interval data: Standardised numerical measurements units like time, weight and temperature are interval data Most informative and accurate form of measurement Increments on the scale can be measured, and they are equidistant. Tell us how many intervals on the scale each person is from anyone else.

Levels of measurement Level of measurement Central tendency Dispersion
Nominal Mode NA Ordinal Median Range Interval Mean SD

Levels of measurement Complete page 15 in your pack.
On the additional hand out…. Colour code, which statements below refer to each level of measurement Which level of measurement: Identify whether the data in each statement is nominal, ordinal or interval. Complete activity C and D.

Choosing the right test

Statistical tests The seven tests that you can be asked about (but won’t have to calculate) in the exam: Chi-squared (χ2) Wilcoxon T Mann-Whitney U Spearman’s Rho Unrelated t-test Related t-test Pearson’s r How do you know which test to use? You need to learn the following criteria, but there is no need for you to understand why this is the case

Criteria Difference or correlation?-Whether the researcher is testing for differences between groups (i.e. an experiment) or a correlation between two co-variables Level of measurement (nominal, ordinal, interval and ratio) In a test of difference, whether the experimental design is an independent groups, repeated measures or matched pairs

LOOKING FOR A CORRELATION REPEATED MEASURES DESIGN
What test to use? Looking for a difference LOOKING FOR A CORRELATION DATA LEVEL INDEPENDENT GROUPS DESIGN REPEATED MEASURES DESIGN NOMINAL Chi-square Sign Test Chi square ORDINAL Mann-Whitney U test Wilcoxon test Spearman’s rho INTERVAL OR RATIO Unrelated t test Related t test Pearsons r

Test of difference or correlation
At Interval level At Interval level Nominal or at least ordinal level data? At ordinal level Pearson’s r Repeated measures Independent groups design Spearman’s Rho Related t test Chi square Chi can also be association. Correlation between chewing gum and having nice breadth. independent groups design? Repeated measures/matched pairs or independent groups design? Mann-Whitney U Test Wilcoxon T Related t test

Chi-Squared Chi-Squared = nominal data = independent groups design

Independent groups Design
Mann-Whitney U Test- Independent women Rank first in the charts Independent groups Design Ordinal data

Wilcoxon T Test Wilcoxon T Test Repeated measures- At least ordinal (1st in the race) Cok sits at the front. Repeating movement. Want to come first.

Spearmans Rho There is a correlation between spearmints chewing gum and fresh breath

Unrelated t-test When you are single, you are independent and Unrelated to a partner, so you have an interval from household chores

Related t-test Back with your partner…. The interval is over… Being related to someone else means…. Repeatedly having to do the household chores

Pearson’s r Brighton Pear is correlational to the west pier. The interval between is made up of a pebbly beach and too many tourists in the summer.

Test of difference or correlation
At Interval level At Interval level Nominal or at least ordinal level data? At ordinal level Pearson’s r Repeated measures Independent groups design Spearman’s Rho Related t test Chi square Chi can also be association. Correlation between chewing gum and having nice breadth. independent groups design? Repeated measures/matched pairs or independent groups design? Mann-Whitney U Test Wilcoxon T Un Related t test

Looking for a difference Looking for a correlation
What test to use? Looking for a difference Looking for a correlation Independent groups design Nominal Interval or Ratio

LOOKING FOR A CORRELATION REPEATED MEASURES DESIGN
What test to use? Looking for a difference LOOKING FOR A CORRELATION DATA LEVEL INDEPENDENT GROUPS DESIGN REPEATED MEASURES DESIGN NOMINAL Chi-square Sign Test Chi square ORDINAL Mann-Whitney U test Wilcoxon test Spearman’s rho INTERVAL OR RATIO Unrelated t test Related t test Pearsons r

Plenary: Now you need to justify each test
Fill in the gaps The Spearman’s Rho was used because the data can be treated as at least 1)_______________ and the researchers were studying a possible 2)_________________ between two co-variables 1 = Ordinal 2 = Correlation (or relationship)

Now you need to justify each test
Fill in the gaps The Chi-Square test was used because the data can be treated as 1)_______________ and the researches had hypothesised that there will be 2)___________________ between conditions when using the 3) _________________________ design. 1 = Nominal 2 = a difference 3 = Independent groups (please note that the Chi-square is also used as a test of association)

Fill in the gaps The Wilcoxon T test was used because the data can be treated as 1)_______________ and the researches had hypothesised that there will be 2)___________________ between conditions when using the 3) _________________________ design. 1 = ordinal 2 = a difference 3 = Repeated Measures (please note that the Wilcoxon T is also used for a matched-pairs design)

Fill in the gaps The Mann-Whitney U test was used because the data can be treated as 1)_______________ and the researches had hypothesised that there will be 2)___________________ between conditions when using the 3) _________________________ design. 1 = ordinal 2 = a difference 3 = independent groups

Test your understanding!
Using your newly found knowledge identify the test that would be suitable for the following: An experiment with nominal data and an independent groups design Ordinal data on both measures in a study to see if two measures are associated An experiment with and independent groups design in which the DV is measured on a ordinal scale A study using a correlational technique in which one measure is interval and the other is ratio. An experiment in which all participants were tested with alcohol and without alcohol on a memory test An experiment in which reaction time was tested using an independent subject design

ANSWERS An experiment with nominal data and an independent groups design chi-squared test Ordinal data on both measures in a study to see if two measures are associated Spearman’s rank correlation / Rho An experiment with and independent groups design in which the DV is measured on a Ordinal scale Mann-Whitney U test A study using a correlation technique in which one measure is interval and the other is ratio Pearsons r An experiment in which all participants were tested with alcohol and without alcohol on a memory test Wilcoxon’s T test An experiment in which reaction time was tested using an independent subject design Unrelated T-Test

Which test to choose. Activity
Draw the diagram/table of inferential statistics Complete pages 19-21

Lesson 3

Choosing a statistical test: complete task E
Test of difference Test of association Unrelated design (Independent) Related design (repeated) or correlation

Choosing a statistical test
Test of difference Test of association Unrelated design (Independent) Related design (repeated) or correlation Nominal data Chi‐square Sign test Ordinal data Mann‐ Whitney Wilcoxon Spearman’s Rho Interval data Unrelated t‐test Related t‐test Pearson’s r

Which test to choose. Activity
Complete the following pages in your pack to test your understanding Pages – question 1-6 extension complete all the questions Pages 27-30

Lesson 4

The process of analysing scientific data
The researcher collects data by carrying out a study The researcher selects the correct statistical test to analyse the data The researcher uses the test to calculate an observed/calculated value The researcher then compares the observed value with a critical value from an already existing table. The researcher states which hypothesis they will retain based on the information in the table

Finding the critical value
the number of participants you used (known as N) apart from for Chi Squared where it is the degrees of freedom (df) which, for your purposes, will always be 1. whether the research hypothesis was one tailed (directional) or two tailed (non-directional). The level of significance you are choosing – which will usually be p =

Scoffing Cheesy Chips Will Make Someone Rather Understandably Porky
Picking the right test Difference Correlation Related data (Repeated measures, matched pairs) Independent data (independent groups design) Nominal Sign test Chi squared Chi-squared Ordinal Wilcoxon Mann-Whitney Spearman’s rho Interval related t-test Unrelated t-test Pearson’s r The tests in bold italics are the ones in which the observed/calculated value has to be ≤ the critical value in order to be significant (accept experimental/reject null) and they form an L for less than to help you to remember. Mnemonic to remember the order: Scoffing Cheesy Chips Will Make Someone Rather Understandably Porky

Practice interpreting inferential statistical tests
Complete separate handout first In your research method packs - complete pages 31-32

Practice interpreting inferential statistical tests Page 33
Independently Practice interpreting inferential statistical tests Page 33

Writing up a statement of significance
What you need to include The test carried out and why The calculated value The critical value N= P<0.05 One tailed test Complete pages 35-37

Lesson 5

Practice exam questions

Key Terms for Statistical Analysis
Probability Psychologists look at data to see if the pattern of results could have occurred by chance. If there results did not occur by chance then we say they are significant. Significance You need to have a null hypothesis (H0) and an alternative hypothesis (H1). What we are looking for is a significant (large) difference in results so that the differences seen in our samples are different and not due to chance; we want to accept the alternative hypothesis. Chance Normally psychologists set the probability level a p≤0.05 which means there is a 5% possibility the results occur by chance in the sample, when there was no real difference in the results in the general population. Observed value The rho or u value calculated is called the observed value. Critical value You need to look in a table of critical values to see if the results are significant. You need to know the 1) degrees of freedom (df) – normally the number of ppts in a study (N); 2) one- or two-tailed test; 3) significance level – normally p≤0.05

Research methods.

Similar presentations

Presentation on theme: "Research methods."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Research methods.

Similar presentations

Presentation on theme: "Research methods."— Presentation transcript:

Similar presentations

About project

Feedback