1. Reliability
Module 6
Activity 5

2. Overview
In this presentation, you will learn about reliability. The following topics will be addressed:
What is reliability?
Different types of reliability
Calculating internal consistency

3. What is reliability?

4. Reliability
Reliability means repeatability or consistency. A measure is considered reliable if it gives us the same result over and over again.

5. Reliability
Reliability is a necessary but not sufficient condition for validity. Remember, validity is the extent to which a test measures what it is intended to measure. For a test to be valid, its results must be reliable. But it must also meet other conditions as well.

6. Reliability
Reliability is associated with statistics. It is a number that can range from 0.00 (unreliable) to 1.00 (absolutely reliable).

7. Reliability
Reliability refers to the results we get from a test or measurement instrument, not to the measurement instrument itself. For example, imagine that I give a reading test to a class of children and the reliability is .23 (very low). I give the same test to a class of teenagers and the reliability is .98 (high). I would say that the results of the test are not reliable for the class of children, but they are very reliable for the class of teenagers.

8. Different types of reliability

9. Different types of reliability
There are three types of reliability: Consistency over time Consistency across equivalent/parallel forms Internal consistency

10. Consistency over time
Consistency over time is also known as test-retest reliability. It answers the following questions: Are test scores reliable over time? Would students get the same score if I gave them the same test in a week?

11. Consistency over time
To examine test-retest reliability, you can give the same students the same test at two different times. Their scores from the first time and the second time should be about the same.

12. Consistency over equivalent forms
Consistency over equivalent forms is also known as parallel form reliability. It answers the following questions: Are the test scores reliable over different forms of the same test? Would the student get the same score if given a different form of the test? When we say form we are referring to a physical test. A different form of the test means that the two tests are assessing the same information, but the items are different.

13. Consistency over equivalent forms
To examine parallel form reliability, you can do the following:
Make two tests that are the same difficulty.
Give both tests to the same group of students.
Compare their scores. Their scores on both tests should be about the same.

14. Internal consistency
Internal consistency refers to how well the test items work together to provide a consistent measurement. It answers the following questions: Is the test internally consistent? Are the items consistent with each other?

15. Internal consistency
There are different ways to measure internal consistency. The most commonly-used method is called Cronbach’s Alpha.

16. Calculating internal consistency

17. Internal consistency
In the next several slides, you will learn how to calculate internal consistency using Cronbach’s Alpha.

18. Cronbach’s Alpha
Go to languagetesting.info

19. Cronbach’s Alpha
Click on “statistics”

20. Cronbach’s Alpha
Scroll down until you see “Cronbach’s Alpha” on the left. Click on “Excel Download” Open the file.

21. Cronbach’s Alpha
You will see an Excel spreadsheet that looks like this.

22. Cronbach’s Alpha
Now, you can enter your data. Enter your students’ names. Enter a “1” if they got the first item right. Enter a “0” if they got the item wrong.

23. Cronbach’s Alpha
Now, look at the bottom of the screen. You should see this: Click on “Results”

24. Cronbach’s Alpha
You will see something that looks like this:

25. Cronbach’s Alpha
This will tell you the average (mean) of the test. It will tell you the standard deviation (SD). And, it will tell you Cronbach’s Alpha (alpha). Cronbach’s Alpha for this test is 0.75.

26. Cronbach’s Alpha
So, how high should reliability be? Remember, the range is between 0.00 and For teacher-made tests, reliability should be about 0.60 – For high-stakes tests, reliability should be higher.

27. Assignment #4
Write a one to two paragraph response to the following questions. This is an off-line, non-graded assignment. Why is reliability important? Why should we expect to see estimates of reliability?

28. Key Terms & Concepts
New Terms: Reliability Test-retest reliability Parallel form reliability Internal consistency

29. Next Steps
Now you should do assignment #5. In this assignment, you will update your assessment glossary.