Aoife Kilduff Rating Scale Research

Rating Scale Usage – 6 point Scale Both Frequency and Development scale rating usage are very similar. In these scales the 4 th and 5 th rating are the most common. The average standard deviation is approximately 1 and there is a bias for positive rating with 84% of ratings being 3 or over. The agreement scale is very different. In this scale the 5 th rating is the most common with respondents choosing ‘Agree’ nearly 50% of the time. The average standard deviation is 0.85 and the bias for positive ratings is very strong with 94% of ratings being 3 or over.

6 point frequency vs 6 point development scale Both the development and frequency scales differ significantly from the normal distribution. There is only a slight difference in the average skewness of the rating scales and they are both biased towards positive ratings with 84% of ratings being 3 or more. However the frequency scale received a negative kurtosis value which suggests that the distribution is flatter than normal. On the other hand the development scale displayed a positive kurtosis value which suggests a distribution which is more peaked than normal. This suggests that in the frequency scale the scores are more spread out.

6 point frequency scale: Labels vs No Labels 6 point frequency rating scales with only end labels are more negatively skewed than scales with all labels present. End label scales display even more of a bias for positive ratings with 87% of ratings being 3 or over compared to 84% on normal 6 point frequency scales. Research also suggests that fully labelled scales are more effective as they ensure that respondents understand the meaning of each point on the scale.

Rating Scale Usage – 5 point Scale For all 5 point scales, there is a bias for the 4 th rating. This bias is significantly more pronounced in the agreement scale where 50% of ratings are ‘Agree’. Although all the scales are negatively skewed, the development scale is the least so with only 59% of ratings being on the positive end of the scale (4 and 5). The frequency rating scale distribution is very negatively skewed with only 6% of ratings being on the negative end of the scale (1 and 2).

Rating Scale Usage – 4 point Scale Again we see that both development and frequency rating scale usage is similar. In these scales the mode is 3 and there is a bias for positive ratings with around 80% of ratings being 3 or over. However in the agreement scale there is a significant bias for the 4 th rating “Strongly Agree” with over 50% of the ratings falling in this category. The agreement scale is also significantly negatively skewed with over 95% of ratings being 3 or over.

Which Scale is the best? Number of Points Type of ScalePro’sCon’s 6 Point Scale (Forces Respondents to ‘make a choice’) FrequencyMost frequently used If wanting to use 6 point scale it is the least negatively skewed There is still a strong bias for positive ratings Using only end labels leads to increase in skewness DevelopmentUseful when survey purpose is developmental Strong bias for positive ratings AgreementSuitable to use with most itemsNearly all responses are positive and there is a very strong bias for the 5 th rating (Agree) 5 Point ScaleFrequencyUseful for items relating to how often the subject demonstrates a certain skill or behaviour Very negatively skewed, only 6% of responses are at the negative end of the scale. DevelopmentLeast negatively skewed overall, good spread of ratings Has been used only a handful of times on our surveys. AgreementSuitable to use with most itemsNegatively skewed and bias for 4 th rating with over 50% responses being ‘Agree’ 4 Point Scale (Forces Respondents to ‘make a choice’) FrequencyLeast negatively skewed if want to use 4 point scale Used very infrequently. DevelopmentUseful when want a clear cut view on development needs Negatively skewed and again used infrequently AgreementSuitable to use with most itemsOver half of all responses are the last rating “Strongly Agree”. Nearly all responses are positive N.B. There is no right or wrong scale to use. What works best depends on the purpose of the survey and the types of items included.