Measurement: A Rasch Analysis of Malaysian Automotive Quality Management-Cost of Quality Scale (MAQM-CoQ Scale) Muhammad Shahar Hj Jusoh , PhD Rushami Zien Yusoff, PhD Mohammad Harith Amlus, PhD Mohd Salleh Hj Din, PhD Azrilah Abdul Aziz, PhD 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Structure of Discussion Overview of Measurement Rasch Model Instrument Construct Findings and Discussion Summary Statistics Principal Component Analysis (PCA) & Misfits Data Person-Item Map Characteristic Curve Causal Relationship Effect Conclusion 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Overview of Measurement Theory of Scales of Measurement (S.S.Stevens, 1946) Law of comparative judgment (Thurstone,1952) Scaling and measurement (Guttman, 1950;1954) Scientific measurement- Thurstone (1952) Estimation of person and item parameters (Rasch, 1961;1968 & Andersen, 1977) Inference- Wright (1998) Additive conjoint measurement- Preece (2002) 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Introduction to Rasch Model Rasch offers a new paradigm in longitudinal research. Rasch is a probabilistic model that offers a better method of measurement construct hence a scale. Rasch gives the maximum likelihood estimate (MLE) of an event outcome. Rasch read the pattern of an event thus predictive in nature which ability resolves the problem of missing data. Hence, more accurate. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Why Rasch Model ? What are the advantages of doing a Rasch analysis? Results easy to read and clearer to understand A parameter estimate (personal profile) for each of the individuals from the data. Comparisons between individuals become independent of the instrument used. Comparisons between the stimuli (items) become independent of the sample of individuals. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Why Rasch Measurement Model ? These leads to: Probabilistic models. Separability of parameters. Parameterization in a multiplicative or additive frame-of-reference. Evaluation of the goodness of fit of the data to the models. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Why Rasch Measurement Model ? When do you need Rasch Analysis? Data in hand is ordinal hence qualitative; but study requires quantitative analysis. Study call for correlation of items. Sample size dealt with is small. A valid scalar instrument of measurement. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Difficulty of a given task Rasch Measurement Model Theorem Two (2) propositions appears: 1. Persons who are more able / more developed have a greater likelihood of correctly answer all the items / able to complete a given task. 2. Easier items / task are more likely to be answered correctly by all persons. In summary: Person Ability Pr (Success ) Difficulty of a given task = - 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Rasch Measurement Model Theorem The Rasch Model incorporates an algorithm that expresses the probabilistic expectations of an item ‘i’ and person ‘n’ performances: Pni(xni=1 | n , i ) = e(n – i ) 1 + e (n – i ) where: Pni(xni=1 | n , i ) is the probability of person n on item i scoring a correct response (x=1); given the person ability, n and item difficulty, i . 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Introduction to Rasch Measurement Model R E Q U I R EM E N T O F M E A S U R E M E N T WHAT IS THE INSTRUMENT USED? WHAT IS THE UNIT OF QUANTITY? WHAT IS THE SCALE CONSTRUCT? IS IT OF LINEAR EQUAL INTERVAL? IS THE MEASURE REPLICABLE? IS IT PREDICTIVE ? 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Rasch Model ‘logit’ scale In Rasch Model, a turn of event is seen as a chance; a likelihood of happenings hence a ratio data.(Steven, 1946) 10 90 10-2 -2 30 70 60 40 50 99 1 100 102 2 -1 exp logit Now, we already have a SCALE with a unit termed ‘logit’. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Rasch Analysis Capabilities INSTRUMENT RELIABILITY RESPONSE VALIDITY CALIBRATION PRINCIPAL COMPONENT ANALYSIS (PCA) QUALITY CONTROL QUANTITATIVE - S.D, Cronbach’s-α, Mean, Z-test PREDICTIVE MODEL 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 1: Summary Statistics Bond and Fox (2007) and Fisher Jr. (2007), analyzing the summary statistic is the essential steps in determining the reliability, validity, consistency and significance issues during the item construct processes. In the Rasch’s summary statistic, prescribes of all the necessary statistical features of sufficient details to represents and answers the goodness of data issues such: reliability, validity, consistency and significance. - Summary Statistics Table 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 1: Summary Statistics -ve Person mean μ = -0.03 logit P[Ɵ] LOi= 0.4921 0.66 ‘Poor’ Person separation of 2 groups. 0.31 ‘Poor’ reliability Valid Responses: 99.9% Cronbach-α :0.33 Poor reliability assessment of student learning 0.99; ‘Very Good’ instrument reliability in item measuring student learning ability 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 1: Summary Statistics 1. Validity Measuring what is supposed to be measured in the right hierarchy (Wright & Stone, 1979; Linacre , 2004). Item reliability will suggest proven evidence on the item difficulty hierarchy is the best terminology to represent the understanding of validity in instrument construct (Andrich, 1988; Bond & Fox, 2007; Fisher, 2007; Linacre, 2008). Validity has no meaning by itself but is useful tools for crafting a variable and defining its numerical properties (Wright & Stone, 1979). 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 1: Summary Statistics 2. Reliability Person reliability used a linear interval scale (when the data fit the model requirements) compared with a usually nonlinear raw score scale used in the calculation of KR-20 (Schumacker & Smith. Jr., 2007). Need to be deliberated by the person separation index as well (Andrich, 1988; Linacre, 2008; Pagani & Zanarotti, 2008; Fisher. Jr. et. al, 2010). 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 1: Summary Statistics 3. Significance How high must the probability be before an investigator is willing to declare that a relationship between variables exists (Ary et.al., 2002) or “the statements that have high probability of being correct rather than an absolute truth statements”. The applicable level of significance test is set at 0.05 level where the Z-standard (normal curve) condition acceptable range is within -2 < Z < +2 in two-tailed test (Andrich, 1988; Bond & Fox, 2007; Fisher, 2007; Linacre, 2008). 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 2: PCA & Quality Control Yen (1993) and Zenisky, et al. (2003) suggested using the local item dependence to detect dependency between pairs of items or persons. Wright (1999), share more than half of their random variance suggesting that only one of these two items is needed for measurement (one of the items have to be discarded or removed). Local dependence would be large positive correlation, with highly locally dependent items (Correlation > 0.7) suggesting that only one of the two items is needed for measurement based table Principal Component Analysis : Largest Standardized Residual Correlations (Wright et al.,2003). 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 2: PCA & Quality Control 4 criteria as to check for any outliers or misfits data, as any misfits pattern to be considered are focused on the requirements given, that are: 1) Point Measure Correlation (PT-Mea Corr); 0.4 < PT-Mea Corr value < 0.85. 2) Point Measure Correlation (PT-Mea Corr); gave a negative value (meaning that the person expected to be misfit due to careless respond or guessing or the given pattern is reverse than the ideal pattern). 3) Outfit Mean Square (MNSQ); 0.5 < Outfit MNSQ value < 1.5 4) Outfit Z-Standard (Z-STD); -2 < Outfit Z-Std value <+2 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 3: Person-Item Map 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 4: Characteristic Curve 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Finding 5: Causal Relationship Effect 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Novelty Developed the measurement ‘ruler’ Measurement Standard Transform ordinal into equal interval scale Measure item or tasks difficulty Measurement Standard Meet the standard hence measurement requirement Validation of instrument construct Better reflect measure of ability Precision and Accuracy of measurement. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Conclusion Rasch probalistic model offers an better method to verify the validity of a measurement construct hence precision. Rasch predictive ability resolves the problem on the need of students taking all the tests; Rasch estimate the likely responses based on anchored items. Rasch gives the maximum likelihood estimate (MLE) of an event outcome. Rasch offers a new paradigm in operational management research via longitudinal research; clearer to read, easy to understand. 2nd International Postgraduate Conference on Business Management (IPCBM 2016), 16-17 February 2016, Pulau Langkawi

Thank you. Any Q ?