A Performance Comparison of Color Vision Tests for Pilots Requirement Marie Marechal Dears professors, dears colleagues, I am pleased to present you my study on the evaluation of color vision tests and their application for pilots requirement. These study was realized with French military health service, at Principal Center of pilots medical examiner

Conflicts of interests : None

Introduction Color-deficiency 9 % of men and 0.5 % of women Protanomalous trichromats Anomalous Deuteranomalous trichromats trichromats Tritanomalous trichromats Protanopes Dichromats Deuteranopes Tritanopes S cone monochromatism Achromatism Color deficiency affects 9% of men in Caucasian population. The main types of hereditary color deficiency are protanomalous and deuteranomalous trichromatism and protanopia and deuteranopia for dichromats 3

Introduction Color use in aviation PAPI (Precision Approach Path Indicator) is the more critical Other are less problematic : Parking lights Taxiways Regulatory lights of airplanes. The color is very used in airports. PAPI: Precision approach path indicator, is the most critical

Introduction Color is more and more present in cockpits, "glass cockpits“ : LCD screens with electronic process of many instruments of flights, whose display is multicolored. Colors choice : free to the manufacturer Color is also more and more present in cockpits, with the development of glass-cockpits. These screens present the flight instruments with a multicolored display. colors choice is free for the manufacturer, but they must be easily recognizable by color-normal subjects.

Introduction - rules In France : Decree of 27 January 2005 and follow EASA regulations - pass the Ishihara 24 plate test (first 15 plates identified without error or hesitation) - ‘normal trichromat’ at the Nagel anomaloscope - identification without mistake or hesitation of colored lights in Beyne lantern (presented at 5 meters for 1 second with 3 minutes of arc aperture) - matching range is 4 scale units or less to the Nagel anomaloscope Normal Color vision Color vision Safe In France, the rules of civil aviation are elaborated according to the European regulation (European aviation safety regulation) Color vision is considered normal when reading the first fifteen plates of Ishihara album (without error or hesitation) ... Or when diagnosed normal trichromatism at the Nagel Anomaloscope If unsuccessful , color vision is considered safe in case of identification of the Beyne lantern lights, or if the matching range is less than 4 units to the Nagel anomaloscope

Introduction - rules Secondary tests in other countries In France : Decree of 27 January 2005 and follow the EASA regulations - pass the Ishihara 24 plate test (first 15 plates identified without error or hesitation) - ‘normal trichromat’ at the Nagel anomaloscope - identification without mistake or hesitation of colored lights in Beyne lantern (presented at 5 meters for 1 second with 3 minutes of arc aperture) - matching range is 4 scale units or less to the Nagel anomaloscope Secondary tests in other countries In UK : Holmes-Wright lantern or CAD test In Germany : Anomaloscope In USA : Farnsworth and Optec lanterns, Titmus vision test, Cone Contrast Test In Canada : lantern test or Farnsworth D15 In New Zeland : Farnsworth and Holmes-Wright lanterns, Farnsworth D15 or CAD test Normal Color vision Color vision Safe   Secondary tests used in other countries are often different: The Holmes-Wright, Farnsworth and Optec lantern, Fifteen Hue are the most used Cone Contrast test and CAD test are new computerized tests

Introduction Lanterns : no more commercialized (only Fletcher-Evans CAM lantern) Aim of the study To assess the abilities of 8 color vision tests for screening and quantifying red / green hereditary deficiency. The problem is the no more commercialization of lanterns. Only the Fletcher-Evans CAM lantern is available ... The aim of the study is to assess the abilities of 8 color vision tests for screening and quantifying red / green hereditary deficiency,  

Methods Prospective study Percy military Hospital, in Clamart (France). INCLUSION Color-deficient subjects: addressed for a failure in reading the Ishihara Color vision normal subjects : healthy volunteers. EXCLUSION BCVA less than 6/10. Sunglasses or tinted contact lenses. Ophthalmologic pathology (evolutionary or sequelar). This prospective study was conducted in Percy military hospital Color vision deficient subjects were addressed for failure in reading the Ishihara plates, The control subjects were healthy volunteers 9

Methods - color vision test Ishihara Beyne lantern Fletcher lantern Farnsworth D15 Arrangements test Lanthony 15 Hue Munsell 100 Hue CAD (Colour Assessment and Diagnosis) test Anomaloscope Nagel type 8 color vision tests were evaluated

Methods - color vision test Ishihara 38 plates Out of order at 70 cm of distance, at 45° 3 seconds for each plate  successful if the first 17 plates were viewed without error or hesitation.   The Ishihara plates Were presented out of order Candidates had 3 seconds to read each plate. The test is successful if the first Seventeen plates are read without error or hesitation

Methods - color vision test Ishihara Beyne lantern Fletcher lantern Farnsworth D15 Arrangements test Lanthony 15 Hue Munsell 100 Hue CAD (Colour Assessment and Diagnosis) test Anomaloscope Nagel type

Methods - color vision test Beyne lantern type aviation simple colored lights, in random order any hesitation or false answer = an error 2 protocols without cited colors 1/25th s / 2 : presentation 1/25th second with 2 minutes of arc aperture 1 s / 4’ : presentation 1 second with 4 minutes of arc aperture 2 protocols with colors cited (red, green, blue, yellow-orange and off-white) 1 s / 3’ : presentation 1 second with 3 minutes of arc aperture 1 s / 3’ x 3 presentations (pass if no error on 2 of 3 presentation) military The Beyne lantern presents 5 colored lights. Any hesitation or wrong answer is considered an error ...... 2 protocols were made without mentioning colors. -       the protocol used for military pilots, with presentation One Twenty-fifth of a second and 2 minutes of arc aperture  - and The protocol used for non-flying militaries, with presentation of each light 1 sec with 4 minutes of arc aperture Then was the protocol used for civil pilots, where colors to recognized are cited: red, green, blue, yellow-orange, off-white The lights are presented 1 second with an 3 minutes of arc opening. According to the french protocol (Civil Aviation Medical Council), 2 additional presentations are made in case of error after the first serie. The candidate is fit to fly if he pass either the first presentation either 2 of 3 presentations… In this study, we evaluated results after the first presentation, or after the 3 presentations civilian

Methods - color vision test Ishihara Beyne lantern Fletcher lantern Farnsworth D15 Arrangements test Lanthony 15 Hue Munsell 100 Hue CAD (Colour Assessment and Diagnosis) test Anomaloscope Nagel type

Methods - color vision test Fletcher-Evans CAM lantern two lights, vertically 5 colors : 2 Red, 2 Green, 1 White 9 combinations Informative phase : colors presented and cited 'red', 'green' and 'white ‘. test phase : 0.9 minute of arc , 2 seconds First round : pass if no error Two more rounds if errors  The Fletcher lantern has 2 vertical lights and 5 different colors : 2 shades of red, 2 shades of green and 1 white. During the informative phase, the colors are showed and named to the subject. During the test phase: the lights are presented 2 seconds. The candidate is FIT if the first passage is successful. In case of error, two additional presentations are made. We evaluated the results after the first presentation and the results after the 2 retests

Methods - color vision test Ishihara Beyne lantern Fletcher lantern Farnsworth D15 Arrangements test Lanthony 15 Hue Munsell 100 Hue CAD (Colour Assessment and Diagnosis) test Anomaloscope Nagel type 3 arrangement tests were evaluated

Methods - color vision test Arrangement tests Farnsworth D15 to classify the 16 pawns in a time of 3 minutes. successful if no confusion right (circular scheme) a straight between the pawns 7 and 15, a simple pawns inversions. Desaturated Lanthony 15 Hue successful if less than 2 lines of confusion Farnsworth-Munsell 100 Hue 3 minutes /box x 4 axis of the deficiency severity score For the Farnsworth Fifteen Hue, the candidate must rank the 16 pawns in 3 minutes. The test is considered successful if no confusing line is present (except for a straight between 7 and 15). The Lanthony desaturated 15 Hue is considered successful if less than 2 confusion right are present For the 100 Hue test, the candidate must rank the pawns in less than 3 minutes per box. This test provides a determination of the color deficiency axis, and a severity score.

Methods - color vision test Ishihara Beyne lantern Fletcher lantern Farnsworth D15 Arrangements test Lanthony 15 Hue Munsell 100 Hue CAD (Colour Assessment and Diagnosis) test Anomaloscope Nagel type

Methods - color vision test The CAD Test (Colour Assessment and Diagnosis) Mesopic condition at 1.40 meter in front of the viewing screen, 3 steps 'learning mode’ : check the test understanding. 'fast screening’ : identify a large part of the healthy subjects. ‘definitive CAD’ : determine the chromatic sensitivity of the subject present for each wavelength stimuli of varying intensity. determines the axis and severity according to a score (RG for the Red-Green axis and YB for blue-yellow axis) Healthy subjects : score < 2 SN Ability according to UK threshold : RG < 6 SN for protans and RG< 12 SN for deutans The CAD test is a computerized test where the subject must determine the direction of the colored pattern movement. There is 3 steps: -           The learning mode check the subject understanding. It presents stimuli seen by all, even color deficient (presented in the movie above) -      the      Fast screening identify most of healthy subjects. It presents stimulus only identifiable by healthy subjects (as presented in the movie down)           In case of too many errors in the fast screening, the subject realizes the “definitive CAD ". This phase determine the chromatic sensitivity, The CAD test determines the color deficiency axis and provides a severity score: Healthy subjects score is less than 2 units. We used in our study the standards proposed by British Civil aviation authority : less than 6 units for deutans and less than 12 units for protans.

Methods - color vision test Ishihara Beyne lantern Fletcher lanter Farnsworth D15 Arrangements test Lanthony 15 Hue Munsell 100 Hue CAD (Colour Assessment and Diagnosis) test Anomaloscope Nagel type

Methods - color vision test The anomaloscope IF2, Tomey Rayleigh equation, on the dominant eye in mesopic conditions 2 modes : Automatic or Manual identifies the deficiency axis, differentiates anomalous trichromats from dichromats, or class the subject in normal color vision matching range  severity Safe : if matching range less than 4 units at the Nagel anomaloscope 100 85 70 60 50 40 30 15 0 The anomaloscope The subject visualized a disc separate in two parts: on the left a mixture of red and green, and on the right a yellow. The subject indicates the mixtures of red and green for which the 2 parts appear to him identical. Accepted Red / Green mixtures are plotted on a graph. the wider is the matching range, the stronger is the deficiency. On the top graph you can see results for dichromats : protanopes in red and deuteranopes in green. On the bottom graph is a deuteranomalous trichromat in green, a protanomalous trichromat in red and a healthy subject in orange. The candidate is considered fit for class 1 if the matching range is less than 4 units 100 85 70 60 50 40 30 15 5

Methods 2 groups : CVD : color vision deficient CVN : color vision normal For each test, evaluation of : Sensibility, Specificity, Positive and negative predictive values Application to aeronautic : Total number and success ratio for class 1 ability Severity assessing : ROC curves for dichromatism diagnosis for CAD test and 100 Hue Subjects were classified into 2 groups: deficient color vision, and normal color vision, after confrontation of the results of all the tests For each test, have been evaluated: Sensitivity, Specificity, and Predictive Value for color deficiency screening their application to aeronautics with the number and success ratio for class 1 ability Severity assessing was evaluated by ROC curves of CAD test and 100 Hue for dichromatism diagnosis

Results 52 subjects 29 VCD 23 VCN p 23 years ± 6.09 26 ± 6.1 years 0 woman 9 women p = 0.015 - 11 deuteranomalous trichromats (37.9%) - 7 protanomalous trichromats (24.1%) - 6 protanopes (20.69%) - 5 deuteranopes (17.24%) Twenty nine color deficient subjects and twenty three control subjects were included There were 11 deuteranomalous trichromats, 7 protanomalous trichromats, 6 protanopes and 5 deuteranopes in the VCD group

Results Detection of color-deficiency tests power Regarding detection of color deficiency The Ishihara test has a excellent sensitivity and specificity

Results Detection of color-deficiency tests power The arrangement tests are very specific but less sensitive

Results Detection of color-deficiency tests power Beyne lantern is also very specific and less sensitive with 1 second presentation * But the protocol used for military pilots, with shorter presentation, is very sensitive but not specific  

Results Detection of color-deficiency tests power Fletcher's lantern has a high sensitivity and is also specific, especially when 2 retests are used

Results Detection of color-deficiency tests power  The CAD test and the anomaloscope in manual mode both have a sensitivity and specificity of 100% * The anomaloscope made in automatic mode is a little less efficient.

Results Class 1 ability for CVN All subjects fit : anomaloscope, CAD test, 15 Hue Ishihara : 2 subjects unfit 100 HUE : 1 subject unfit (low discrimant) Beyne lantern : 1/25th s / 2’ : 11 fail on 23 (47.8%) 1s / 4’ : 1 fail 1s / 3’ : all pass Fletcher lantern : 5 subjects fail on first presentation, pass with retest Regarding subjects with normal color vision .... They were all fit for class 1 according to the anomaloscope, the CAD test, and both 15 Hue tests. 2 healthy subjects were unfit according to Ishihara album, and One subject according to 100 Hue . Concerning the Lantern Beyne, military pilots protocol for 1/25 s / 2 was more selective with nearly fifty % failure. For Fletcher's lantern, 5 subjects from the VCN group made a mistake during the first presentation, and all corrected their mistake with the first retest … So subjects with normal color vision may fail some tests

Results Total number and success ratio for a class 1 ability for CVD In the CVD group ... .. The number of subjects fit for a class 1 is very variable according to the test used

Results Total number and success ratio for a class 1 ability for CVD The Ishihara album, the Fletcher lantern and the Beyne lantern protocol used for military pilots are the most selective.

Results Total number and success ratio for a class 1 ability for CVD The Lanthony 15 Hue desaturated , the Beyne lantern with presentation 1 second whithout citing colors, and the anomaloscope, allow an aptitude to 20% of CVD The CAD Test is more restrictive and allows an aptitude for 10% of CVD.

Results Total number and success ratio for a class 1 ability for CVD The 100 Hue and the Beyne lantern civil protocol allow the ability of 20 to 30% of CVD subjects. The standard 15 Hue test is the most tolerant and allows a professional pilot certificate to more than 40% of color deficient subjects

Results Total number and success ratio for a class 1 ability for CVD * For these three tests, at least one dichromate subject (therefore with severe damage) were fit to fly ... ..

Results CVD Ishihara D15 15HUE Lanthony. 100 HUE BEYNE lantern FLETCHER lantern CAD test Anomal. 1/4 1/25th 2 1/3 3 tests 1 test 2 retests 1 fail PASS 2 Fail 4 5 6 7 11 14 15 16 17 20 22 24 25 26 27 29 30 31 Results Regarding results details for CVD subjects, we can be seen that the success of one test does not guarantee to pass other tests. 35

Results CVD Ishihara D15 15HUE Lanthony. 100 HUE BEYNE lantern FLETCHER lantern CAD test Anomal. 1/4 1/25th 2 1/3 3 tests 1 test 2 retests 1 fail PASS 2 Fail 4 5 6 7 11 14 15 16 17 20 22 24 25 26 27 29 30 31 Results * For example, some subjects can only pass the Beyne lantern according to the current civil protocol and fail all other tests ... 36

Results CVD Ishihara D15 15HUE Lanthony. 100 HUE BEYNE lantern FLETCHER lantern CAD test Anomal. 1/4 1/25th 2 1/3 3 tests 1 test 2 retests 1 fail PASS 2 Fail 4 5 6 7 11 14 15 16 17 20 22 24 25 26 27 29 30 31 Results  Other subjects pass only the standard test of 15 Hue There is therefore a high variability of test results in these CVD subjects 37

Results Deficit quantification AUC: Best cut-off: 0,98 [0,94 ; 1,01] for CAD test 0,65 [0,47 ; 0,83] for score difference in 100 Hue 0,69 [0,52 ; 0,86]for 100 Hue CAD test 100 Hue : difference between achieved score and physiologic score 100 Hue : achieved score Best cut-off: RG > 18,5 for CAD test Score difference > 22 Score > 112 for 100 Hue For tests with severity score, we performed ROC curves for dichromatism diagnosis (taking anomaloscope as a reference). the CAD test has an area under the ROC curve much higher than the 100 Hue test (either the score alone or the difference between the score obtained and the physiological score of the subject). For CAD test, the best cut -off to separate dichromats of abnormal trichromats subjects is a score more than eighteen point five (18.5)  

Discussion Squire et al. in 2005, UK 3 lanterns (Beyne, Spectrolux, and Holmes-Wright) and Nagel anomaloscope. 55 CVD subjects and 24 CVN subjects. all dichromats failed the 4 tests. pass one secondary test : not guaranteed to pass the other tests  Tests authorized by the EASA standards : high variability and inconsistency of their results. ,,,,,In the previous study, Squire evaluated the results of 3 lanterns and anomaloscope. In Squire's study, as in ours, passing one secondary test don’t guarantee to pass the other tests. The authors therefore conclude that the tests authorized by the European standards show a high variability. Squire TJ, Rodriguez-Carmona M, Evans ADB, Barbur JL. Color vision tests for aviation: comparison of the anomaloscope and three lantern types. Aviat Space Environ Med. 2005 May;76(5):421–9.

Discussion Fletcher in 2005 Similar to Holmes-Wright lantern : 2 lights with a 0.9 minute arc aperture. BUT color used : slight differences in the CIE diagram. 9 / 71 normal trichromats : mistakes in first pass. 18 color-deficient subjects : all failed. Fletcher-Evans CAM lantern : very sensitive.  good test for clinical diagnosis, not for chromatic selection. Birch in 2008 Holmes-Wright lantern type A 125 color-deficient subjects: 10 subjects (9%) were able to pilot Nevertheless, Fletcher lantern not comparable to the Holmes-Wright lantern (passing 9 to 30% of CVD) To our knowledge, only one study evaluated the Fletcher-Evans CAM lantern, Like in our study, all color deficient subjects failed, and some of normal trichromats failed the first pass. This lantern is very sensitive. This is a good clinical diagnostic test, but it seems less useful for professional chromatic selection. Fletcher lantern is not comparable to the Holmes-Wright lantern which allows ability for 9 to 30% of CVD - Fletcher R. The Fletcher CAM lantern colour vision test. Optom Today. 2005 Jul;(29):24–6. - Birch J. Performance of colour-deficient people on the Holmes-Wright lantern (type A): consistency of occupational colour vision standards in aviation. Ophthalmic Physiol Opt. 2008 May;28(3):253–8.

Discussion Walsh in 2016 , USA 65 CVD subjects and 68 CVN subjects of US Army. Our results : similar except a better diagnostic efficiency of the CAD test.  good efficiency of the computerized tests CCT and CAD operator-independent and randomized patterns Test Sensitivity Specificity CAD test 0.86 0.85 to 1.00 Farnsworth D15 0.35 1.00 PIPIC 0.98 0.96 to 1.0 Walsh study evaluated several color vision tests including the CAD test , With similar results as ours They also evaluated Cone Contrast Test, with a better efficiency than the CAD test. The authors conclude that the CAD test has good efficacy, which has the advantage of being non-operator-dependent and presenting randomized patterns. Walsh DV, Robinson J, Jurek GM, Capó-Aponte JE, Riggs DW, Temme LA. A Performance Comparison of Color Vision Tests for Military Screening. Aerosp Med Hum Perform. 2016 Apr;87(4):382–7.

Discussion British CAA, 2006 117 CVD Ability of PAPI lights recognition and CAD test results No subject who pass CAD test failed to PAPI simulator. (pass : RG < 6 SN for protan or RG < 12 SN for deutan) CAD test : pass in 36.1 % of deuteranomalous trichromats 29.8% of protanomalous trichromats In our study : only 18.2% deuteranomalous trichromats 14.3% protanomalous trichromats A study by the British Civil Aviation Authority assessed the PAPI lights recognition on a simulator, and CAD test results. No subject who passes the CAD test failed the PAPI simulator This study found a success of the CAD test in more than 30% of color deficient In our study we found lower success rates. (The difference could be explained by a lower rate of mild deficiency in our study. ) Barbur JL, Rodriguez-Carmona M, Evans S, Milburn N. Minimum Colour Vision Requirements for Professionnal Flight Crew. Recommendations for a new colour vision standards. Safety Regulation Group. Civil Aviation Authority; 2009.

Discussion New Zealand CAA, 2015 Decrease the requirements for color vision - Information coding multiple - colors choose to be recognized by CVDs PAPI simulator used by City University : not representative of real PAPI CAD test : too restrictive USA FAA , 2015 Color deficient subject : same recognition of PAPI lights as color normal The Civil Aviation Authority of New Zealand recommend to reduce the requirements for pilots’ color vision , (They criticize the PAPI Simulator used by City University, which would not be representative of the real PAPI ) According to them, the CAD test standards proposed by the British authority would be too restrictive. This report quotes an American study which found that color deficient subjects had the same performances as healthy subjects for PAPI simulator recognition - Brookes RE. Colour Vision Requirements for Aircrew [Internet]. New Zealand Defence Force. Dfence Technology Agency; 2015 [cité 16 août 2017]. Disponible sur: http://www.dta.mil.nz/wp-content/uploads/Report405.pdf - Milburn N, Gildea K, Perry D, Roberts C, Peterson L. Usability of Light-Emitting Diodes in Precision Approach Path Indicator systems by Individuals With Marginal Color Vision [Internet]. Civil Aerospace Medical Institute Federal Aviation Administration; 2014.

Discussion No learning Simple and reproductible Test protocol strictly respected and trained examiners  can limit anomaloscope use Short time Ishihara, lanterns and 15 Hue : < 5mn 100 Hue and anomaloscope : about 20mn CAD test : < 5mn if "Fast screening " + 8mn for "Definitive CAD "  : 6 CVN (26%), all CVD A test adapted to professional selection should not be learned, and must be simple and reproductible Test protocols must be strictly followed, and the examiners trained, which may limit a large-scale use of anomaloscope. The test duration is also important in this screening context. Ishihara's album, lanterns and 15 Hue tests are fast, less than 5 minutes. The 100 Hue test and the anomaloscope take about 20 minutes. The CAD test timing is variable: less than 5 minutes if the Fast screening is successful, but about 15 minutes if definitive CAD, which was the case for some control subjects .

Discussion CAD test Lanterns Advantages Advantages Can’t be learn Reproducible Red / green and yellow / blue axis 16 colors Quantitative test Advantages Quick Easy Less expensive Ergonomic test The CAD test has the advantage of being non operator-dependant, quantitative and evaluate the red / green and blue / yellow axis. lanterns have the advantages of being fast and easy , and less expensive.

Conclusion Many tests are used for pilots ability. Their results are discordant Ishihara : first intention test Lanterns are no more commercialized CAD test and anomaloscope : most accurate in our study Recent increase of colored signals in new generation cockpits. Multitude of color amount. Risk of not detected blue-yellow deficiency. The acceptable color-deficiency for a pilot to be safe is now to determine. IN CONCLUSION Many tests are used for pilots ability, but their results are discordant . Ishihara's album is an excellent screening test for red / green deficiency but don’t detect blue / yellow deficiency Beyne lantern would be no longer available. Fletcher lantern seems too restrictive The CAD test and anomaloscope are the most efficient in our study, and they are quantitative tests   The recent increase of colored signals of in cockpits and the diversity of colors used, leads to questioning the current selection protocols. The current non-detection of blue-yellow deficiency could be a risk to aviation safety. The acceptable chromatic deficit to guarantee the safety of flights remains to determine.

Thank you for your attention

Discussion In conclusion Ishihara plates : excellent screening test for red / green deficiency. Farnsworth D15 : quick and easy but some dichromats subjects can pass this test. Lanthony 15 Hue test : ability of 20% and seems suitable for use in professional selection. Munsell 100 Hue : too long Fletcher-Evans CAM lantern : too restrictive Beyne lantern : simple and quick to use, but some dichromats pass. Anomaloscope : qualify and quantify the deficiency requires a strong experience of the examiner. CAD test : quantitative test , yellow/blue and red/green thresholds used by the British CAA seem to be more restrictive More expensive

Results Assessment of the axis of the color-deficiency