1. 1 ESREL 2003 European Safety and Reliability Conference June 15-18, 2003 - Maastricht, the Netherlands Assessing Part Conformance by Coordinate Measuring Machines Daniele Romano University of Cagliari (Italy) – Department of Mechanical Engineering Grazia Vicario Politecnico of Turin (Italy) – Department of Mathematics

2. 2 Problem Study of uncertainty of industrial measurement processes and its implications on process design Objectives Analysis of uncertainty in position tolerance check of manufactured parts on Coordinate Measuring Machines Optimal allocation of the measurement points on the part surfaces Problem and objectives

3. 3 The research area (Metrology, Statistics, Engineering Design) Analyze Uncertainty Design better Product/Process Regulations & Standards Measurement Instrument Measurement Process Methods & Techniques Simulation Monte Carlo simulation DOE Computer Experiments Robust Design Statistical Inference... Product/Process Objectives Driving force

4. 4 Errors in Measuremen t SYSTEMATIC RANDOM  Orthogonality errors between slides  Form errors of slides  Non-linearity of amplifier response  Errors due to the approach angle of the touch-ball  …. What’s a CMM? Inherent sampling error

5. 5  Planes A, B, C, definfìing the reference system, are ideal mating surfaces which real part surfaces are contacted with in the referencing order (A first, then B, then C)  Nominal hole axis is perpendicular to datum A and displaced by Xc and Yc from datum C and B respectively.  Actual hole axis isthe axis of the ideal largest size pin able to enter the hole perpendicular to plane A The hole location problem

6. 6 Our measurement process 1. Estimation of datum A (envelope to the part surface) 2. Estimation of datum B (envelope) 3. Estimation of datum C (envelope) 5. Probing points on the hole surface 6. Projection of points on datum A 4. DRF origin is obtained by intersection of the three datums 7. Estimation of the largest size inscribed circle

7. 7 Calculating position error X Y DRF origin Plane of datum A Xc Yc Measured points projected on datum A Inscribed circle Cnom Cact epep

8. 8 Acceptance rule Deterministic Probabilistic  e peq  =  e p   (d act  d min )/2  t/2  an uncertainty measure Identifier of Maximum Material Condition (MMC)

9. 9 Outline of the study The real measurement process is replaced by a stochastic simulation model (Romano and Vicario, 2000). In the model:  Measurement errors on the coordinates returned by the CMM are considered additive and described by i.i.d. normal random variables with zero mean and common variance,  2 = 0.005 2 mm 2.  The part has no error. Experimentation is conducted on the simulation model investigating how uncertainty in the measure of the position error is affected by the number of points probed on the surfaces (control factors) and by part geometry (blocking factors). A Monte Carlo simulation (N=10 4 ) is run at each experimental trial to have a reliable estimate of uncertainty.

10. 10 The experiment Dimensions in mm Simulation model Number of points probed on each surface Random error Device variables Measurand geometry uncertainty in the measure of position error

11. 11 array Patterns of measurement points On planesOn hole surface Helix

12. 12 The uncertainty measure A convenient representation for position error is the polar one, e p =  e i  and a suitable measure of uncertainty for e p is the area of a conjoint confidence region I of the two-dimensional random variable (  ) at a (1-  ) level, defined as: A useful way to solve the integral is by using conditional distribution f  and marginal f  : A numerical solution is then provided by taking equally sized angular sectors and using the empirical distributions f  and f  (deriving from Monte Carlo simulations).

13. 13 0.005 0.01 0.015 0.02 30 210 60 240 90 270 120 300 150 330 1800 [mm] Uncertainty depends on the angle of the position error Finding Proposal of a different acceptance rule Consequence  e peq  (m)  t/2  P  (  e peq  /  (m)  Empirical 95% confidence region of e peq for two experimental settings Solid boundary: all factors at high level Dashed boundary: all factors at low level

14. 14 Empirical 95% confidence region of e peq for two experimental settings Solid boundary: most polarized Dashed boundary: least polarized 0.005 0.01 0.015 30 210 60 240 90 270 120 300 150 330 1800 [mm] Polarization depends on factors

15. 15 Role of Xc and Yc 0.005 0.01 0.015 0.02 30 210 60 240 90 270 120 300 150 330 1800 [mm] Solid boundary: Xc = Yc = 50 mm Dashed boundary: Xc = Yc = 0 mm (All other factors are at the low level)

16. 16 Factorial effects on uncertainty Finding Allotment of measurement points on the surfaces as adopted in industrial practice is not optimal. As an example, quota of points on the datums A, B, C are based on the 3:2:1 rule, disproved by results. Best allotment also depends on the part geometry. Consequences Effects on A 95 Normal score

17. 17 Designing efficient measurement cycles Given a prediction model for uncertainty a simple optimization problem can be defined in order to find the allotment of probed points on the surfaces ( x ) that minimizes uncertainty for a given part ( b 0 ) and a given total number of probed points ( n TOT ): x = (n H n A n B n C ) T b 0 = (w 0 Xc 0 Yc 0 d 0 ) T LB and UB are bounds on x where

18. 18 Two design examples b 0 = (75mm 100mm 100mm 50mm) T A quadratic response surface for is estimated from the experiment ( ) and used for optimization The part geometry is defined by: Solution is sought for in the experimental range: LB = (4 4 4 4) T UB = (16 16 16 16) T [m2][m2] [  m] Results

19. 19 Conclusions A statistical analysis of position error as measured by CMM has disproved a number of engineers beliefs:  Tolerance zone is a circle  Acceptance rule contains only the modulus of position error  The number of measurement points on planar datums A, B, C is best decided according to the 3:2:1 rule  The best allocation of measurement points on the surfaces does not depend on part geometry (plate thickness, boxed dimensions) A L F S E A L F S E A L F S E A L F S E A comprehensive analysis of uncertainty is a prerequisite for an efficient design of the measurement process. Statistical methods and computer simulation seems a unique combination to cope with it.

20. 20 Scientific work on uncertainty in CMM measurements Most of the work addresses the characterization of measurement errors due to the machine and the related calibration methods to compensate systematic errors. The basic scenario for uncertainty analysis has been proposed by PTB and then adopted also by other metrology Institutes. In the approach the first measure is taken by the real machine, all other are obtained via a computer simulation model ( “virtual machine”). We are not aware of applications of uncertainty analysis on the design of an efficient measurement process. Practitioners routinely select measurement cycles by applying simple rules of thumb where cost is the major concern.

21. 21 P C Q K O O’ Plate thickness role in position error Absolute reference Datum Reference Frame C: nominal position of hole center on DRF Case #1 Plate thickness = h 4 points probed P = estimated center position PC position error Case #2 Plate thickness =2 h 4 points probed Q = estimated center position QC position error Case #3 Plate thickness = 3h 4 points probed K = estimated center position KC position error

22. 22 z x y 3D plot of the origin of the Datum Reference Frame 270.000 points Uncertainty depends on direction

23. 23 0.0020.0040.0060.0080.010.0120 0 5 10 15 20 25 30 35 40 45 Frequency  =2,5° Case of the most polarized 95% confidence region 200 400 600 800 30 210 60 240 90 270 120 300 150 330 1800 Frequency

24. 24 0.0020.0040.0060.0080.010.0120 0 5 10 15 20 25 30 35 40 45 Frequency  =2,5° Case of maximum polarized 95% confidence region 0.005 0.01 0.015 30 210 60 240 90 270 120 300 150 330 1800 [mm]

25. 25 Uncertainty Analysis Basic Product/Process Design Take the same measurement N times Estimate uncertainty of that measurement Take M measurements according to an experimental design Replicate the experiment N times Estimate uncertainty in the whole sampling space Knowledge of uncertainty and cost in the sampling space Select hardware components Select parameters of the measurement process Design specifications (uncertainty, cost) Comprehensive

26. 26 Planar datums in the referencing order with orthogonality constraint (Orthogonal Least Squares + shift) and estimation of the origin of the Datum Reference Frame (DFR) Hole axis (Orthogonal Least Squares) Position error (distance between nominal and actual axis) in DRF Monte Carlo simulation on the ideal parts (ideal form, perfect dimensions) with a measurement error   N(0,  2 ),  2 = 0.005 2 Study of the dependence of uncertainty of origin of DFR on the number of the inspected points on the surfaces through a 3 3 experimental design Position Tolerance Check and its Uncertainty on CMM Estimation of features Methodology Evaluation of uncertainty of position error

27. 27  i x +  i y +  i z +  i = 0with i = 1,2,3 Mathematical models Estimation of planar datums and origin of DRF Position Tolerance Check and its Uncertainty on CMM Probed points on surfaces Ref. A Ref. C Ref. B

28. 28 Steps Maximum Likelihood estimators of parameters Orthogonal Least Squares Non-linear problem let use a constraint (Lagrange multiplier) Equivalent problem with Solution: unit norm eigenvector associated to the minimum eigenvalue First Datum Position Tolerance Check and its Uncertainty on CMM

29. 29 Maximum Likelihood estimators of parameters Orthogonal least Squares + orthogonality constraint with the first datum Same problem as the first datum unit norm eigenvector associated to the minimum eigenvalue... Steps Second Datum + Third Datum Position Tolerance Check and its Uncertainty on CMM

30. 30 Step Origin of DRF Position Tolerance Check and its Uncertainty on CMM Envelope rule

31. 31 Results: scatterplots of the origin of the DRF Origins of estimated datums as envelope surfaces Origins of estimated datums with Orthogonal Least Squares 9 inspected points on actual surfaces Position Tolerance Check and its Uncertainty on CMM  Envelope rule, when form errors are comparable with measurements errors, produces a bias and increases uncertainty  Uncertainty depends on direction

32. 32 Why does uncertainty depend on direction? Position Tolerance Check and its Uncertainty on CMM OLS lines with orthogonality constraint OLS lines with no constraint Orthogonality constraint makes a pattern!

33. 33 100 200 300 400 30 210 60 240 90 270 120 300 150 330 180 0 00.0050.010.0150.02 5 10 15 20 25 30 35 40 45 50 Frequency  =135° Frequency Position Tolerance Check and its Uncertainty on CMM d(C nominal.,C actual )=f( ,  ) Dependence on direction suggests to express position error by a polar (spherical) transformation in the two dimensional case

34. 34 Results : 95% Confidence Regions 0.01 mm 30° 210° 60° 240° 90° 270° 120° 300° 150° 330° 180° 0° 0.02 mm 0.03 mm Position Tolerance Check and its Uncertainty on CMM  =0.005 mm  Measurement error is largely amplified  Reduction of uncertainty is heavily paid in terms of number of measurement point n 1 =n 2 =n 3 =4; n c =4 n 1 =n 2 =n 3 =9; n c =9

35. 35 Results: effect of the number of measured points on the flat surfaces on uncertainty (of origin of DRF) Position Tolerance Check and its Uncertainty on CMM with O = (X O, Y O, Z O ) DRF origins A-optimality with a 3 3 experimental design

36. 36 Amount of uncertainty in the estimation of position error is not negligible and it may easily leads to incorrect decision about acceptance/rejection of the part, if not considered Uncertainty depends on direction: a non trivial software module should be added to the machine Results suggest some criticism of the envelope rule: The tolerance zone (including uncertainty in the evaluation) looses the central symmetry Envelope rule is unjustified and detrimental (biased estimates and increased uncertainty) when form errors of inspected surfaces are comparable with random error of CMM Final Remarks Position Tolerance Check and its Uncertainty on CMM

37. 37 Position Tolerance Check and its Uncertainty on CMM CMM gives: I.coordinates of a finite number of points pertaining to contact points between a touch probe and the planar datums according to a specific order coordinates of a finite number of points pertaining to contact points between a touch probe and the hole surface CMM software computes coordinates and gives parameters “estimates” of probed surfaces, but the current practice does not include any uncertainty evaluation Measurements process with CMM