MEASURING PROCESS QUALITY ON AN ORDINAL SCALE BASIS E. Bashkansky, T.Gadrich Industrial Engineering & Management Department E.Godik Software Engineering Department

2 Presentation Outline I. Introduction II. Objective III. Basic Definitions IV. Previous Results: pro et. contra V. Proposed Models of a Quality Ladder VI. Checking Properties of the Models by the help of Simulation VII. Conclusions

3 PROCESS QUALITY Gap Action Process Quality Control Target Measure

4 Two types of data characterizing products or processes Variables (results of measurement on basis of numerical Interval or Ratio scale ) Attributes (results of testing, estimation on basis of Nominal or Ordinal Scales ).

5 Ordinal Variables in Quality Engineering Quality sort Customer satisfaction grade Vendor’s priority Customer importance (QFD) Failure severity Internet page rank Vote result (pro, abstain, contra) the power of linkage (QFD) … Traditional approach: assigning arbitrary numerical values to the different categories of the ordinal variable

6 It is necessary to know: Specification Requirements: Location of the Process – μ – process average Variation of the Process – σ 2 – process dispersion How the quality of a stable process is measured on the numerical scale basis ?

7

8 Quality variable having three levels of quality Traditional Approach Quality level Assigned value H9 M3 L1 Quality level Assigned value H3 M2 L1 “H” – High Quality “M” – Medium Quality “L” – Low Quality H > M > L Scale AScale B

9 Traditional Approach - Average Sample HLL  According to A latent scale the average equals 1.67 positioning the average between Low and Medium quality  According to B latent scale the average equals 3.67 positioning the average between Medium and High quailty

10 Variation measure HMM vs. MLL HMMMLL A 0.33 B

11 Study’s Purpose Estimation the quality of a stable process without assigning any numerical values to the ordinal variables.

12 Median HHHHHHMMMLLLL Advantage: Simple Natural Measure for the Ordered Samples Disadvantage : R obust HHHMLLLMMMMLLL

13 How the quality of a stable process is measured on the ordinal binary scale basis ? “G” – Good Quality “B” – Bad Quality quality = 1- proportion of defective units q = 1-p

14 How the quality of a stable process is measured on the ordinal binary scale basis ? “1” – Good Quality “0” – Bad Quality quality = 1- proportion of defective units q = 1-p If p is a proportion of defective units in a process,q- is a proportion of good units in a process, then a number of defectives- d in a random sample n from infinite population:

15 Sample proportion of defectives distribution n=10,p=0.3 (q=0.7)

16 Sample proportion of defectives distribution n=100,p=0.3 (q=0.7)

17 Sample proportion of defectives distribution n=1000,p=0.3 (q=0.7)

18 Variation of sample quality measure vs. quality level- q (for n=100) Uncertainty is maximal at a 50% quality level !

19 Quality measure of a given sample Equals to the relative position of the given sample in a quality ladder that is built for a samples of the same size.

20 The Rational of a Quality Ladder Q HH…H quality represented by a sample LL…L

21 Various possible quality ladders for a sample n=2 HH HM MM HL ML LL HH HM HL MM ML LL HH HM HL=MM ML LL

22 The problem starts already from a sample n=2 What is the “right” order in a quality ladder : LL < ML < HL < MM < HM < HH or: LL < ML < MM < HL < HM < HH or: LL < ML < HL = MM < HM < HH

23 F-function (Cumulative distribution function – CDF) Define: P i = proportion of products belonging to i - th quality level. F L = P L ; F M = P L + P M ; F H = P L + P M + P H =1

24 Pareto dominance criterion Ordered sample X Pareto-dominates ordered sample Y, if all elements in Y do not dominate the corresponding elements in X, and at least one element in X dominates the corresponding one in Y, note this as X Y. In the case that no dominance relationship can be defined between sample X and Y, named this as X and Y belong to the same equivalence class and note this by X ≈ Y.

25 Pareto dominance criterion - example For a sample size 3, from all 10 ordered sample, look at the following samples: sample A: HMM; sample B: HHL; sample C: MMM; One can establish that A C ; A ≈ B ; B ≈ C. In general, the Pareto-dominance criterion gives a ‘poor’ ordering for the sample space of an ordinal quality characteristic, because most of samples remain indistinguishable.

26 Stochastically larger criterion ( X is stochastically larger than Y means, that the CDF associated with X lies below (or touches) the CDF associated with Y for all levels of quality. F i Figure 1: Graphical interpretation of the concept stochastically larger 12k

27 Stochastically larger criterion-the previous example LMH level of quality Figure 2: Stochastically lager's examination for ordered sample A, B and C 1

28 The general conclusion: Stochastically larger criterion is equivalent to Pareto dominance criterion

29 Graphical presentation of a different sample using F-space R={LLLLL} S={HHHHH} T={MMMMM} O={HHMLL} P={HMMLL{ Q={HMMML}

30 F-space: Sample vs. population The only difference between sample and population in their presentation on the F – plane is the points density

31 Ordinal dispersion Blair, J., & Lacy, M. G. (2000)

32 Proposed Quality Ladders 1. Rank and dispersion (R&D)- based on Franceschini F., Galetto M., Varetto M., Qual. Reliab. Engng. Int. 2005; 21:177– Median and Entourage (M&E) 3. Proportion Ratio and Dispersion (PR&D)

33 1.Rank and dispersion criterion (R&D criterion) The algorithm has two stages: First stage sorts the samples in ascending order according to their ranks value. Rank value = 0*(# L) + 1*(# M) + 2*(# H) Second stage orders samples belonging to the same rank class according to their dispersion values in descending order. The ordered sample having larger dispersion is located at a lower position in the quality ladder. Disadvantage

34 1.Rank and dispersion criterion (R&D criterion) - example Q

35 Rank & Dispersion criterion : pro & contra Pro – simple Contra: Ordinal scale means that codes assigned to objects represent the rank order of the entities measured. The concept of distance between two generic levels of the same scale is meaningless. Comparisons of greater and less can be made, in addition to equality and inequality. However operations such as conventional addition and subtraction are without meaning. Franceschini et al. definition Rank value = (number of M in the sample) + 2* (number of H in the sample) means a vicarious attempt to convert, so or otherwise, an ordinal estimation to the numerical by assigning of a numerical value to each level of the ordinal scale. ( 0 - to “L”, 1- to “M”, 2 - to “H”)

36 Graphical interpretation of R&D criterion FLFL 1 T S R O 45 0 Figure 4: Graphical illustration of rank and dispersion criterion FMFM

37 2.Median and Entourage (M&E)-recurrent ordering procedure for a samples of size n 1. First, ordering samples according to their median : Odd :L < M < H Even : LL < ML < HL < MM < HM < HH 2. In the second stage, within each equivalence classes the samples are ordered according to their entourage of size n-1(or n-2).

38 2.Median and Entourage Criterion (M&E criterion)-Example Quality

39 Median and Entourage criterion : n = 5 FLFL FMFM Figure 5: order sample size 5 by median and entourage criterion

40 3.Proportion Ratio and Dispersion criterion (PR&D) - first stage Define the proportion ratio (PR) as: As the quality of the sample increases, the value of PR decreases, and vice versa. So, first, samples are arranged according to their decreasing PR values.

41 Graphical illustration of PR criterion FLFL FMFM Figure 6: Graphical illustration for Proportion ratio criterion T S R 1 Z 0 1 Q

42 Proportion Ratio and Dispersion criterion (PR&D) – second stage For the case ( relatively rare), when more then one sample is represented by the point lying on the same straight line,these samples are ordered according to their dispersion in the way similar to Rank & Dispersion criterion

43 Comparison between various criterions for a sample size n=3

44 Comparison between various criterions for a sample size n= 5.

45 Quality measure - definition The quality represented by a sample is characterized by it's relative position in a quality ladder built for a samples of the given sample size.

46 Example

47 Verification of proposed criterions sample (n=100) vs. infinite population

48 Verification of proposed criterions sample (n=100) vs. infinite population

49 Verification of proposed criterions relative position of the sample (n=100) mode quality vs. relative quality position in the infinite population

50 Verification of proposed criterions relative position of the sample ( n=10 ) mode quality vs. relative quality position in the finite ( N=100 ) population

51 Verification of proposed measures: relative position of the sample ( n=10 ) median quality vs. relative quality position in the finite ( N=100 ) population

52 Conclusions

53 Thank You