1. Specialist Registrar in Occupational Medicine
Extent and determinants of agreement between Medical Assessors in the assessment of seafarer fitness
Dr Isam Rustom
Specialist Registrar in Occupational Medicine
22nd November 2011

2. RESEARCH QUESTIONS
What is the extent of agreement between Approved Doctors (ADs) and Medical Referees (MRs) in assessing the fitness of seafarers?
What is the extent of agreement between the MRs in assessing the fitness of seafarers?
What are the determinants of agreement between the MRs in assessing the fitness of seafarers?

3. METHODS
Seafarer Medical Review- Medical Referee’s Report (MSF 4108) forms were collected for the period between May 2003 and May 2009.
MRs were assigned individual codes (1-12) to ensure confidentiality.
The medical categories of those seeking review were coded using the 16 categories on the (MSF 4108) forms.

4. METHODS
Type of work has been categorised as Deck, Engine, Catering and Others.
The AD’s and MR’s decisions were categorised into one of the four fitness categories.
The reasons of grading or restrictions change were coded into four categories:
-More Information
-New Event
-GP/Specialist Report
-Other (which includes experience, knowledge and different interpretation of the standards by the MRs)

5. STATISTICAL ANALYSIS
Standard statistical tools were used: Chi-square, P values < 0.05, inter-rater reliability Kappa statistic and the ration between upgraded/downgraded cases for each MR.
The outcome of the appeal process (result of appeal) for the purposes of analysis is taken to be the difference between the AD’s decision and the MR’s decision which were classified as:
- No change (same category, but could alter the imposed restrictions);
- Upgrade (MR’s decision gave the seafarer more flexibility to work at sea);
- Downgrade (MR’s decision was more strict than the AD’s decision, further limiting the seafarer work at sea).

6. MR’s referrals - age profile

7. Medical categories and mean age

8. AD’s decision vs. MR’s decision
Number
Percentage
MR’s decision
Fit
0.0 66
14.2
Fit with
restrictions
111
23.9
Fit with restrictions
252
54.2
Temporarily unfit 86
18.5 62
13.3
Permanently unfit
268
57.6 85
18.3
Total
465
100.0

9. Reasons for grading and restrictions change
Reasons for grading / restrictions change n (%)
Medical
category
More
information
New event
GP/Specialist
report
Other
Total
Cardiovascular
65 (45.4)
18 (12.6)
27 (18.9)
33 (23.1)
143
Endocrine
20 (30.3)
8 (12.1)
5 (7.6)
33 (50.0) 66
Sensory
10 (16.7)
3 (5.0)
37 (61.6) 60
Neurological
15 (34.1)
1 (2.2)
5 (11.4)
23(52.3) 44
Respiratory
10 (23.8)
1 (2.4)
3 (7.1)
28 (66.7) 42
Mental
17 (44.7)
0 (0.0)
7 (18.5)
14 (36.8) 38
Musculoskeletal
12 (33.4)
4 (11.1)
3 (8.3)
17 (47.2) 36
149 (34.7)
42 (9.8)
53 (12.4)
185 (43.1)
429
*Genitourinary, digestive, blood and cancer (36) cases were excluded from analysis, see Methods section

10. Extent of agreement between the MRs
The workload varied considerably between MRs as they were distributed across the UK.
Referee 9 saw (95 cases),
Referee 6 (68 cases),
Referee 10 (61 cases),
Referee 7 (46 cases),
These Referees dealt with more than half (270 cases; 58%) of all appeal cases over the 6-year period of the study.

11. Ratio of upgraded/downgraded cases by MRs
Dove trait
Hawk trait
Referee code
a,b 6 10 9 1 3 5
Ratio
(upgraded/downgraded) 38 30 20 18
13.5
12.5
a Referee 2,4,7 and 11 had no downgraded cases, so their ratios were 0.
b Referee 8 and 12 were excluded from the analysis as they only considered 5 and 1 respectively and had no downgraded cases, so their ratios were 0.

12. Determinants of agreement between the MRs
Age group
Medical category
Referee code
Type of work

13. There was significant association between age group and “Fit with restrictions” fitness category except under 20s age group.
There was significant association between medical category and “Fit with restrictions” fitness category except Mental ill health cases.
There was No significant association between MR’s decision and Referee code.
There was No significant association between MR’s decision and type of work.

14. Discussion of extent of agreement between ADs and MRs.
This study identified poor agreement between the MR’s decision and the AD’s decision among those seafarers who decided to appeal against the AD’s initial decision [kappa=0.18; poor].
The most frequent category for a grading or restrictions change was “Other reasons” (43.1%), in all medical categories apart from Cardiovascular 65 (45.4%) and Mental ill health 17 (44.7%) cases which "More Information" was the most frequent category when reviewing fitness decisions.

15. Outcome of the appeal process
58% of the appellant seafarers benefited from the process and were granted an upgrade in their fitness category.
40% of the appellant seafarers who had no change in their fitness category often benefited from less or different restrictions which allowed them to continue in employment and working at sea at some capacity.
Only the 2% of the appellant seafarers who had been downgraded were disadvantaged by the process.

16. This study attempted to determine the factors that may have affected the outcome of the appeal process. The relationships with measurable variables like age group, medical category, Referee code, and type of work were studied.
It appears that the outcome of the appeal process was multi-factorial and was affected by a combination of the above variables.

17. CONCLUSIONS
First research question: this study found that there was poor agreement between the MR’s decision and the AD’s decision regarding the fitness of those seafarers who decided to appeal against the AD’s initial assessment.
Second research question: MRs tended to agree with each other as they upgraded the fitness category of seafarers to allow more of them to work at sea.
Third research question: MRs decisions on the fitness of seafarers were affected by the age group and the medical category and MRs preferred to use “Fit with restrictions” fitness category more than any other fitness categories in order to impose more or less restrictions whilst keeping the seafarer in employment.

18. The way forward
In 2008 the MCA decided to review the 2002 fitness standards in the light of changes in medical diagnosis, treatment, disability legislation and decision taking process especially apparent in the reviews undertaken by MRs. As a result more seafarers would be allowed to work at sea with restrictions and reasonable adjustments which were taken into consideration in the new 2010 fitness standards.

19. FURTHER RESEARCH
There is a need to explore other factors which might affect the ADs and MRs decisions when assessing the fitness of seafarers. For example, the hypothesis being that ADs with occupational medicine qualifications would be less likely to have their verdicts set aside on appeal. There might be a gradient, with higher levels of qualifications more likely to produce a supported verdict.

20. Thank You