1. Adopting The Item Response Theory in Operations Management Research
Hj Muhammad Shahar Jusoh, Phd
Keynote Speaker: 2nd International Conference on Business, Science and Technology 2015 (ICBST), April 2015

2. Presentation’s Structure
Overview
Classical Test Theory (CTT) vs Item Response Theory (IRT)
Idea of the Study
Why Adopting IRT
Implementing in Operations Management Research
Conclusion

3. Overview Theory of Scales of Measurement (S.S.Stevens, 1946)
Law of comparative judgment (Thurstone,1952)
Scaling and measurement (Guttman, 1950;1954)
Scientific measurement- Thurstone (1952)
Estimation of person and item parameters (Rasch, 1961;1968 & Andersen, 1977)
Inference- Wright (1998)
Additive conjoint measurement- Preece (2002)

4. Overview
“Social studies will not become science until students of social phenomena learn to appreciate this essential aspect of science”
“Draw conclusions in the presence of uncertainty”

5. Classical Test Theory vs Item Response Theory
- Use the total score to characterize each person.
- “Observed Score (X) = True Score (T) + Error (E)”.
- Total score as the relevant statistic with little consideration of anomalies (variance) in the items or the individuals answering them.

6. Classical Test Theory vs Item Response Theory
Latent Traits Theory / Modern Test Theory
- Based on the idea that the probability of a correct response to an item is a mathematical function of person and item parameters.
- The person parameter is called latent trait or ability.
- Bring greater flexibility and provides more sophisticated information.
Computerized Adaptive Testing

7. Idea of Study
Manuscript from the Philosophical Transactions of the Royal Society in 1763:
“Bayes showed how inverse probability could be used to calculate probability of antecedent events from the occurrence of the consequent event. His methods were adopted by Laplace and other scientists in the 19th century, but had largely fallen from favor by the early 20th century. By the middle of the 20th century, interest in Bayesian methods had been renewed by De Finetti, Jeffreys, Savage, and Lindley, among others. They developed a complete method of statistical inference based on Bayes' theorem”
Bolstad (2007)

8. Why Adopting IRT
1. Treating the raw scores using the scale formatted as interval data (Acton, 2003; Schumacker & Smith, Jr.,2007; Battisti et al., 2010 & Preece, 2002). 2. Develop more accurate applications consecutively to test the item construct for accurate and measurable testing (Ganglmair & Lawson, 2003; Hambleton & Jones, 2005; Pallant et al., 2006; Pana et al., 2007; Gothwal et al., 2009 and Casteleijn, 2010). 3. Rigorous capabilities for instrument construct and validation (Wright & Linacre, 1989; Linacre, 2000, 2001, 2003; Fisher, 2008).
Computerized Adaptive Testing

9. Why Adopting IRT
4. Sophisticated findings in marketing research, advertising research, human resources selection criteria, customer satisfaction and service quality perspectives (Salzberger, 2000; Battisti et al., 2003; Ewing et al., 2005; Salzberger & Sinkovics, 2006). 5. Exhaustive and richness of data representation (Wright & Linacre, 1989; Linacre, 2000, 2001, 2003; Fisher, 2008)
Computerized Adaptive Testing

10. Implementing in Operations Management Research
1. Reliability
Ability of a measure to remain the same; consistently over time or the same result is obtained when the same research is repeated
or does it again once more (Sekaran, 2003).
Cronbach’s alpha is used as the common value in estimating the internal consistencies of the items (Onwuegbuzie & Danial, 2002).
Need to further consider the other reliability index as well.

11. Implementing in Operations Management Research
Person reliability used a linear interval scale (when the data fit the model requirements) compared with a usually nonlinear raw score scale used in the calculation of KR-20 (Schumacker & Smith. Jr., 2007).
Need to be deliberated by the person separation index as well (Andrich, 1988; Linacre, 2008; Pagani & Zanarotti, 2008; Fisher. Jr. et. al, 2010).

12. Implementing in Operations Management Research
2. Validity
Measuring what is supposed to be measured in the right hierarchy (Wright & Stone, 1979; Linacre , 2004).
Item reliability will suggest proven evidence on the item difficulty hierarchy is the best terminology to represent the understanding of validity in instrument construct (Andrich, 1988; Bond & Fox, 2007; Fisher, 2007; Linacre, 2008).
Validity has no meaning by itself but is useful tools for crafting a variable and defining its numerical properties (Wright & Stone, 1979).

13. Implementing in Operations Management Research
3. Significance
How high must the probability be before an investigator is willing to declare that a relationship between variables exists (Ary et.al., 2002) or “the statements that have high probability of being correct rather than an absolute truth statements”.
The applicable level of significance test is set at level where the Z-standard (normal curve) condition acceptable range is within -2 < Z < +2 in two-tailed test (Andrich, 1988; Bond & Fox, 2007; Fisher, 2007; Linacre, 2008).

14. Implementing in Operations Management Research
3. Significance
The direction of differences is not important as long as the value congregate (from the sample mean observed) is tabulated within the range.
The region of rejection is equally divided between the two tails of the distribution thus if
a sample mean is observed that is either sufficiently greater or sufficiently less than the hypothesized value then the assumption or premise would be rejected.

15. Implementing in Operations Management Research
4. Distribution Map

16. Implementing in Operations Management Research
5. Characteristic Curve

17. Implementing in Operations Management Research
6. Causal Relationship Effect

18. Conclusion
Using and choosing a right measurement model will derive towards acquiring research problems hence answering the reliability, validity and significance.

19. Thank you. Any Q ?