Reliability Module 6 Activity 5

Overview In this presentation, you will learn about reliability. The following topics will be addressed: What is reliability? Different types of reliability Calculating internal consistency

What is reliability?

Reliability Reliability means repeatability or consistency. A measure is considered reliable if it gives us the same result over and over again.

Reliability Reliability is a necessary but not sufficient condition for validity. Remember, validity is the extent to which a test measures what it is intended to measure. For a test to be valid, its results must be reliable. But it must also meet other conditions as well.

Reliability Reliability is associated with statistics. It is a number that can range from 0.00 (unreliable) to 1.00 (absolutely reliable).

Reliability Reliability refers to the results we get from a test or measurement instrument, not to the measurement instrument itself. For example, imagine that I give a reading test to a class of children and the reliability is .23 (very low). I give the same test to a class of teenagers and the reliability is .98 (high). I would say that the results of the test are not reliable for the class of children, but they are very reliable for the class of teenagers.

Different types of reliability

Different types of reliability There are three types of reliability: Consistency over time Consistency across equivalent/parallel forms Internal consistency

Consistency over time Consistency over time is also known as test-retest reliability. It answers the following questions: Are test scores reliable over time? Would students get the same score if I gave them the same test in a week?

Consistency over time To examine test-retest reliability, you can give the same students the same test at two different times. Their scores from the first time and the second time should be about the same.

Consistency over equivalent forms Consistency over equivalent forms is also known as parallel form reliability. It answers the following questions: Are the test scores reliable over different forms of the same test? Would the student get the same score if given a different form of the test? When we say form we are referring to a physical test. A different form of the test means that the two tests are assessing the same information, but the items are different.

Consistency over equivalent forms To examine parallel form reliability, you can do the following: Make two tests that are the same difficulty. Give both tests to the same group of students. Compare their scores. Their scores on both tests should be about the same.

Internal consistency Internal consistency refers to how well the test items work together to provide a consistent measurement. It answers the following questions: Is the test internally consistent? Are the items consistent with each other?

Internal consistency There are different ways to measure internal consistency. The most commonly-used method is called Cronbach’s Alpha.

Calculating internal consistency

Internal consistency In the next several slides, you will learn how to calculate internal consistency using Cronbach’s Alpha.

Cronbach’s Alpha Go to languagetesting.info

Cronbach’s Alpha Click on “statistics”

Cronbach’s Alpha Scroll down until you see “Cronbach’s Alpha” on the left. Click on “Excel Download” Open the file.

Cronbach’s Alpha You will see an Excel spreadsheet that looks like this.

Cronbach’s Alpha Now, you can enter your data. Enter your students’ names. Enter a “1” if they got the first item right. Enter a “0” if they got the item wrong.

Cronbach’s Alpha Now, look at the bottom of the screen. You should see this: Click on “Results”

Cronbach’s Alpha You will see something that looks like this:

Cronbach’s Alpha This will tell you the average (mean) of the test. It will tell you the standard deviation (SD). And, it will tell you Cronbach’s Alpha (alpha). Cronbach’s Alpha for this test is 0.75.

Cronbach’s Alpha So, how high should reliability be? Remember, the range is between 0.00 and 1.00. For teacher-made tests, reliability should be about 0.60 – 0.85. For high-stakes tests, reliability should be higher.

Assignment #4 Write a one to two paragraph response to the following questions. This is an off-line, non-graded assignment. Why is reliability important? Why should we expect to see estimates of reliability?

Key Terms & Concepts New Terms: Reliability Test-retest reliability Parallel form reliability Internal consistency

Next Steps Now you should do assignment #5. In this assignment, you will update your assessment glossary.