1. Insert dateInsert placeInsert name E- Quality Assurance Tool A User Guide and Best Practice Cologne, 13-14 November 2007

2. European Aviation Safety Agency 2 Data Entry: A Quality Issue Data entry can involve errors or omissions Eccairs has more than 600 fields with structured and unstructured data Currently, in ECCAIRS there is no context related system to:  Prioritise data input  Check validity of data (data mining)

3. European Aviation Safety Agency 3 Consequences of Errors: GIGO Database reliability drops Analysis becomes inaccurate Not all data is captured Analysis produces “false positive” hits Waste of Resources  Time  Effort  Maybe even…Database

4. European Aviation Safety Agency 4 Quality vs. Information Should records be entered when key data is missing?  Date  Registration  Aircraft type  Location  Number of involved persons Balanced approach is needed: Too many incomplete records will reduce quality of DB Tagging records could help

5. European Aviation Safety Agency 5 Types of Errors Type I : Omissions or no data entered Example: Missing level of injury although accident was fatal Type II : Information entered is wrong Example: Aircraft caught fire after it exited the runway but was coded only as RE

6. European Aviation Safety Agency 6 Type I Errors Data errors  Missing key Data or  Incorrect Data Records will not be retrieved when conducting a search/query Records will be retrieved but will not be used because key values are missing

7. European Aviation Safety Agency 7 Finding Type I Errors  Queries that will search for empty fields Example: Find All Records where Date has no value  Queries searching for empty fields in relationship to other fields Example: Injury level missing when Accident was Fatal

8. European Aviation Safety Agency 8 Preventing Type I Errors  Form filters that accept only specific values or formats (developer only)  Step by step process of inserting records. (example: General, Aircraft, History of Flight etc.)

9. European Aviation Safety Agency 9 Type II Errors Information errors  Data format is right but information is wrong  Usually due to limited expertise in using the database Records will be retrieved although they are NOT related to the scope of analysis Records will not be retrieved although they refer to a related occurrence

10. European Aviation Safety Agency 10 Finding Type II errors Read thoroughly every single report… if you have the courage! Peer review the records in batches Use queries built for identifying disparities  Mismatch Events-Categories ….EQAT

11. European Aviation Safety Agency 11 Use of EQAT for Type I Errors

12. European Aviation Safety Agency 12 Cell Colour denotes importance Colour is for decision support. A high percentage of errors on a non-critical field might be more acceptable than a smaller percentage in a critical one. For example… Colour Coding is not Percentage dependant

13. European Aviation Safety Agency 13 Cell Colour denotes importance Colour is for decision support. A high percentage of errors on a non-critical field might be more acceptable than a smaller percentage in a critical one. For example… Colour Coding is not Percentage dependant

14. European Aviation Safety Agency 14 Level of Importance Mandatory »Yellow 0 to 4 Percent »Orange 4 to 7 Percent »Red 7 to 10 Percent Critical »Yellow 0 to 10 Percent »Orange 10 to 20 Percent »Red 20 to 100 Percent Non-Critical »Yellow 0 to 20 Percent »Orange 20 to 40 Percent »Red 40 to 100 Percent

15. European Aviation Safety Agency 15 Use of EQAT for Type I Errors

16. European Aviation Safety Agency 16 Use of EQAT for Type I Errors Monthly or even weekly checks through the use of global filter

17. European Aviation Safety Agency 17 Global Filter Global filter may be used to limit or broaden the scope of the quality check. It can be set just as any query to limit results according to:  Time: Week, Month, Year, etc.  Report Status: Closed, Preliminary, etc.  State File numbers: 8xxx and 9xxx  Person or Organisation creating the report  …or any other attribute depending on the scope

18. European Aviation Safety Agency 18 Use of EQAT for Type I Errors

19. European Aviation Safety Agency 19 Use of EQAT for Type I Errors

20. European Aviation Safety Agency 20 Use of EQAT for Type I Errors

21. European Aviation Safety Agency 21 Use of EQAT for Type I Errors Run EQAT… before exporting data for other organisations or departments Before sending to JRC or other data repository How to…

22. European Aviation Safety Agency 22 Finding those records Knowing the number of Errors we may use ECCAIRS queries to find the reports containing those errors. The ECCAIRS query will have to match the query used under EQAT…AND the GLOBAL FILTER. (No global filter exist in query libraries) Finding the right query

23. European Aviation Safety Agency 23 Use of ECCAIRS for Type I Errors

24. European Aviation Safety Agency 24 Use of ECCAIRS for Type I Errors

25. European Aviation Safety Agency 25 Use of ECCAIRS for Type I Errors

26. European Aviation Safety Agency 26 Use of ECCAIRS for Type I Errors

27. European Aviation Safety Agency 27 Use of EQAT query library to find and correct Type I errors Specific records may now be checked for missing fields or data. Use of Global filter may help identify records of specific interest (last month, week etc.) Data may be checked before importing/exporting Use of Global filter assist in having a methodical approach to data quality assurance- periodic controls.

28. Insert dateInsert placeInsert name E- Quality Assurance Tool A User Guide and Best Practice Part II

29. European Aviation Safety Agency 29 Finding Type II errors Read thoroughly every single report… if you have the courage! Peer review the records Use queries build for identifying disparities REMEMBER:

30. European Aviation Safety Agency 30 What are we talking about ? 1 - Information is on the record, but the coding is not adequate. 2 – Information is on the record, but not used for coding. 3 – Information is not recorded, although it would be easy to enter - or is dispersed. These are the main non-conformities that we are detecting in the data

31. European Aviation Safety Agency 31 This is what we are talking about ! If the Coding is not correct, it is leading you to false conclusions.false conclusions Correct it ASAP! (1) If the information is there, not coded, you are not using your assets.not using your assets Correct it when you have the time! (2) If it is easy to input, but missing, it will not give you that extra “leverage”.missing Complete it if you can, but do it on new entries! (3)

32. European Aviation Safety Agency 32 The importance of quality review The importance of finding non-conformities depends on: How relevant to Safety is the data? - Is it fundamental to allow us to take conclusions? Example (geographical coordinates) Example (Turbulence – as propeller/jet blast injuries/fatalities !) How frequent is the issue? - Does it happen so often that a Standard Query to find it is necessary?

33. European Aviation Safety Agency 33 The starting approach - relevance How useful is the data? If the data is Category coding, it will give us the capability to capture systematically the Nature and Frequency of a type of occurrence. Then, we can Analyse it and …... take measures to prevent it! If the data is Event coding, it will give us the capability to capture systematically the Circumstances of the occurrence. So, we can also Analyse it and …... take preventative measures! Categories, event type, flight phase... They should be right, because they are basic for Safety Analysis

34. European Aviation Safety Agency 34 The starting approach - frequency How frequent is the non-conformity? If it is fairly frequent, it will prevent you from making a correct analysis of the situation. Therefore, it should be fixed ! If it is rare, it may not make a big impact on a correct analysis. It should also be fixed, but attend to important matters first !

35. European Aviation Safety Agency 35 Starting Approach ? ? ? Why is this a starting approach? As we all gain experience, the picture will change ! We will be systematically entering more relevant and more complete data … We will be assigning the correct Categories, Events, etc. … We will be refining our “error-trapping” … The queries will be adjusted accordingly !

36. European Aviation Safety Agency 36 What do you gain? Will see the forest, not the trees, meaning that you will have a good image of the safety issues affecting your State. With quality data, you: You will be selecting the most important issues to solve and not be lead into chasing minor problems. You will perceive the global situation when receiving experience from other States with the same Quality standard. You will be confident that the data you PRODUCE and RECEIVE is RELIABLE and complete.

37. European Aviation Safety Agency 37 The Library for Errors Type II The Library for Type II errors was developed with the following issues in mind : 1 - Information is on the record, but the coding is not adequate. 2 – Information is on the record, but not used for coding. 3 – Information is not recorded, although it was easy to enter, or is dispersed. And considering the frequency of the errors. Remember:

38. European Aviation Safety Agency 38 The Library Is divided in Themes In the Air On the Ground ATM-related Near the ground Injuries Miscellanea In each theme there are three levels

39. European Aviation Safety Agency 39 Use of the Library At the first time, run all the level one queries, and take note of how many results you get. You are invited to use the Library as follows: If you can handle them all, correct the findings. Otherwise, filter to just one year at a time and correct the findings starting from the more recent years. When you have time, proceed like this with Level 2 and then with Level 3 queries. With the experience gained, knowledge of your resources and workload, establish a Procedure on how often you must run the Queries, and which ones. Keep in mind that you will probably need different timing for Type I and Type II errors and for the different levels.

40. European Aviation Safety Agency 40 Remember ! Read the NARRATIVE thoroughly If the coding of EVENTS is incorrect, the assignment of CATEGORY will probably be inadequate ! And vice-versa … The queries may reveal: Missing assignment of coding Wrong assignment of coding Garbage (why ?)

41. European Aviation Safety Agency 41 Some examples

42. European Aviation Safety Agency 42 Example 1: Ground Collision Cat GCOL not assigned but Event corresponds Find all Occurrences where { Occurrence category. doesn't have any of GCOL: Ground Collision and [ Event phase. has at least one of Taxiing to/from runway, Air taxi/hover taxi, Taxiing: Other and Event type. has at least one of Collision a/c on ground, Collision a/c-object-ground ] } Found n …. (one on next slide ) T2_Lev1_Ground.e4q

43. European Aviation Safety Agency 43 Example 1: Ground Collision THE A/C HAD BEEN PUSHED BACK AND DE- ICED. THE GROUND CREW/DE-ICING CO- ORDINATOR WAS ADVISED TO DISCONNECT THE HEADSET AND USE HAND SIGNALS. THE FLIGHT CREW COULD NOT SEE THE GROUND CREW MAN. AS THE BRAKES WERE RELEASED, THE A/C ROLLED FORWARD STRIKING THE DE-ICING VEHICLE, WHICH WAS NOT VISIBLE FROM THE COCKPIT.

44. European Aviation Safety Agency 44 Example 1: Ground Collision

45. European Aviation Safety Agency 45 Example 2 : Icing Find all Occurrences where { Occurrence category. doesn't have any of ICE: Icing and Event type. equal to Icing conditions } Found n …. Cat ICE not assigned but Event with Icing Conditions T2_Lev1_Air.e4q

46. European Aviation Safety Agency 46 Example 2 : Icing During APP to an A/P in a remote area the aircraft touched down about 2 km before the threshold of the 3.500 ft RWY. Freezing rain was predicted for the area. Ice was found on the leading edge of the wing.

47. European Aviation Safety Agency 47 Can you have your own queries ? These are standard queries, to use if you do not have the time or experience to devise them yourself ! These will be widely distributed, so you will know that others will be on the same level of quality. By all means! If you make your own quality queries, wouldn’t you consider SHARING them with the community ? Please consider it ! We will help by making them available !

48. European Aviation Safety Agency 48 Type II Query titles Ground

49. European Aviation Safety Agency 49 Type II Query titles Near ground

50. European Aviation Safety Agency 50 Type II Query titles Air

51. European Aviation Safety Agency 51 Type II Query titles ATM

52. European Aviation Safety Agency 52 Type II Query titles Injuries Miscellanea

53. European Aviation Safety Agency 53 Quality Queries I & II – Questions ?

54. European Aviation Safety Agency 54

55. European Aviation Safety Agency 55 False conclusions …

56. European Aviation Safety Agency 56 Not using your assets

57. European Aviation Safety Agency 57 Missed opportunities

58. European Aviation Safety Agency 58 Geographic Coordinates