1. Diagnostic accuracy of four respiratory rate timers to detect pneumonia by frontline health workers in sub-Saharan Africa and Southeast Asia
Kevin Baker1,8, Tobias Alfven1, Akasiima Mucunguzi2, Lena Matata3, Emily Danzer4, Tedila Habte5, Jill Nicholson1, Max Petzold6,7, Karin Källander1,8
1Malaria Consortium, UK; 2Malaria Consortium, Uganda; 3Malaria Consortium, South Sudan; 4Malaria Consortium, Cambodia; 5Malaria Consortium, Ethiopia; 6Gothenburg University, Sweden; 7University of the Witwatersrand, South Africa 8Karolinska Institute, Sweden
8122
Key messages
None of the four devices tested performed well based on agreement with the reference standard
The WHO recommended device, the Mark Two Acute Respiratory Infection (MK2 ARI) timer performed at the same level as the other devices tested
Automated, easy to use respiratory rate (RR) diagnostic aids for assessing the symptoms of pneumonia for frontline health workers should rapidly be developed and tested
Suitable, validated reference standards need to be developed for future testing of new pneumonia diagnostics aids.
Results
Most Community Health Workers (CHWs) managed to get a RR count, but agreements at ±2 breaths per minute from the reference standard were low, ranging from 26 percent for the respirometer to 35 percent for the ARI with beads. Performance was consistently lower when assessments were carried out on younger children (0-59 days).
Agreement for RR classification was moderate across all devices, with a Cohen Kappa statistic ranging from 0.41 (SE 0.04) for the respirometer to 0.49 (SE 0.05) for the MK2 ARI. 
Rrate has the highest positive percent agreement (PPA) at 71 percent. MK2 ARI had a PPA of 53 percent, the Respirometer had a PPA of 58 percent and the ARI with beads had a PPA of 44 percent.
MK2 ARI had the highest negative percent agreement at 92 percent.
Introduction Pneumonia is one of the leading causes of death in children under five in both sub-Saharan Africa and Southeast Asia. The current diagnostic criteria for pneumonia are based on increased respiratory rate (RR) in children with cough and/or difficulty breathing. Health workers often find it difficult to accurately count RR and current counting devices are difficult to use or unavailable. This study is the first large scale comparative evaluation of the accuracy of diagnostic aids for low-resource settings.
Methods
Four RR timers were evaluated for accuracy on 601 children over three months by community health workers in hospital settings in Cambodia, Ethiopia, South Sudan and Uganda. These RR timers were evaluated against a reference standard – an automated monitoring device (Masimo Root patient monitoring and connectivity platform with ISA CO2 capnography). As no gold standard existed, agreement statistics, percentage agreement and Kappa values are presented.
Device
MK2 ARI
(n=322)
RRate
(n=304)
Respirometer
(n=626)
ARI with Beads
(n=172)
N ±2bpm
(%)
104
(32)
102
(34)
165
(26) 60
(35)
Kappa value
(SE)
0.49
(0.05)
0.44
(0.06)
0.41
(0.04)
(0.07)
PPA
(%) (95%CI) 53
(41.9, 63.5) 71
(59.1, 80.3) 58
(51.3, 64.9) 44
(29.5, 58.8)
NPA
(%) (95% CI) 92
(87.6, 95.0) 78
(72.4, 83.5) 82
(78.3, 85.9) 93
(86.7, 96.6) a b c d
Figure 1: Devices tested: a) MK2 ARI timer; b) Beads and ARI timer;
c) Respirometer feature phone application;
d) Rrate SMART phone application.
Kevin Baker is currently Programme Coordinator – pneumonia diagnostics at Malaria Consortium in London and is completing doctoral studies at Karolinska Institutet in Stockholm on pneumonia diagnostics.
Karolinska Institutet
Department of Public Health Sciences
Global Health – Health Systems and Policy Research Group
Tomtebodavagen 18A,
17177 Stockholm
Sweden
Malaria Consortium
Development House
56-64 Leonard Street
London EC2A 4LT, U.K. •
Telephone:
Website: