1. An Evaluation of Open Source GIS Routing Tools in Direct Vaccine Delivery in Kano State, Northern Nigeria Kehinde A. Adewara, Snr. GIS Coordinator Presentation to FOSS4G Seoul 2015 Conference 18 September 2015

2. ACKNOWLEDGMENT AND CONTRIBUTIONS Before I begin permit me to start my presentation with acknowledgment and contribution. It is absolutely necessary that I thank you all for finding time to come, to every one who had contributed in every little ways to the success of the FOSS4G Seoul 2015 conference, for the travel grant award I have received as well as selfless support from the Management and staff of eHealth Africa (eHA), Nigeria.

3. Outline 1)Introduction 2)Justification for the Research 3)Conceptualization 4)Approach/Methodology 1.Drive Test Survey 2.Desktop Routing Estimation 3.Multi Criteria Ranking 5)Research Outcomes 1.Desktop Routing Outputs 2.Comparative Outcomes of Error Margin 3.Scoring/Ranking 6)Research Findings, Conclusion and Recommendations

4. Introduction The enormous challenges associated with vaccine delivery to support routine immunization in northern Nigeria have been tackled by government and development partners. Such immense efforts need to be complemented with a measure that is cost effective and efficient in view of dwindling resources. Hence this article took advantage of the enormous benefits of open source resources to addressing some of these challenges. The comparative capability of selected number of open source GIS routing tools were investigated in this regards. 1 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

5. Justification for the Research This research was influenced by the 1.The need to eliminate software license cost and optimize vaccine delivery activities using reliable, credible and cost effective open source routing tools 2.Need to take advantage of emerging open source GIS routing tools, 3.Need to build confidence in the use of open source GIS routing tools as a result of proliferation 4.Need to embrace open source GIS routing tools from a cautious, pragmatic and objective perspective (Graser, Straub, & Dragaschnig, 2015). 5.Need to determine the most appropriate routing tools in view of its consequence on decision making outputs. 2 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

6. Conceptualization Drive test and desktop routing techniques were based on shortest path concept or graph theory which considered travel time as constrains or impediment. It is evident that some applications use Euclidean route calculation as the basis for routing (fig. 1). The output from this calculation may not be reflective of ground reality. Hence spatial route calculation helps to address this limitation (fig. 2). Fig. 1 - Euclidean Route Fig. 2 - Spatial Route 4 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

7. Approach/Methodology In determining the most appropriate open source GIS routing tools, the following techniques were adopted 1.Drive Test Survey, 2.Desktop Routing Estimation and 3.Multi Criteria Ranking 3 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

8. Drive Test Survey Drive test survey has been used in different studies as a control measure depending on study objective and scope. It is widely used in telecommunication studies (Sanders, Linder, Pratt, Dickherber, Floyd, & Pickering, 2004; Boxberger, Lawver, & Smithey, 2010). However it was used in this case as a technique for acquiring baseline information which serves as a benchmark for determining accuracy among different routing tools. It may be practically impossible to conduct drive test survey for all the facilities in the state due to logistic challenge, hence a representative sample size of 326km was considered with minimum error of margin using an online sample calculator. The sample size was determined at 95% confidence level and narrow confidence interval (CI) of 5 (Myles, Douglas, & Eric, 2013). 5 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

9. Drive Test Survey – Cont’d The sample population (437km) was determined as proximity/distance between the state cold store and the farthest facility in each zone (table 1). Hence, approximately a buffer distance 85.74km above the sample benchmark (326km) was covered in the survey. The actual distant covered was 411.74 km for 10 facilities. Table 1 - Sample Distribution 6 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

10. Drive Test Survey – Cont’d The travel distance and time during the drive test survey was determined using mobile mapping GPS enabled solution called OsmAnd tablet device (fig. 3). The device is made up of a detailed base map, navigation tool and a plugin called trip recording. The plugin is the app used for recording distances covered during the drive test survey. 7 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools Fig. 3 – OsmAnd Interface

11. Drive Test Survey – Cont’d Drive test and routing estimation were conducted to determine actual travel distant measurement between the state cold store and the 10 selected health facilities (table 2). Table 2 – Kano State Primary Health Facilities targeted for Routing Destinations 8 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

12. Desktop Routing Estimation The desktop routing estimation was used to compare routing outputs from the five (5) open source GIS routing tools (Haklay, 2010). It was conducted for all the delivery routes connecting the 10 facilities during the drive test survey. These tools were primarily classified as online, mobile and desktop routing tools (see table 4). Table 4 - GIS Routing Tools 9 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

13. Desktop Routing Estimation (cont’d) The delivery route is an optimum route (with shortest possible distance) connecting the state cold store to the 10 health facilities across the state. Certain assumptions were made and certain limitations were unresolved. It is assumed that all routing tools consider travel by vehicle and fastest (not shortest) path option. 10 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

14. Desktop Routing Estimation (cont’d) Limitations include inability to handle traffic conditions as well as inability to consider same routes among all the routing tools due to insufficient base map. These limitations and other factors (geo positioning accuracy, limitations of inbuilt routing/network model) were the reasons for the observed discrepancies in the routing outputs using same base map (see figure 4). Figure 4 - Different Outputs from Routing Tools using same base map 11 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools

15. Multi Criteria Ranking In view of the limitation associated with using error margin condition between drive test and routing estimation outputs as a sole factor for determining best routing tools, it is imperative to consider other ‘win and loose’ advantages associated with these tools. There are several methods of conducting such ranking well documented in the literature, most common is the additive weighing factor approach (Tofallis, 2014). Multiplicative weighing score was able to overcome certain limitations with additive weigh. It is commonly used in the decision making hierarchy of world leading organizations such as UNDP in developing annual HDI, an instrument used for ranking nations based on human per capita income, life expectancy and education (UNDP, 2010). 12 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Open Source GIS Routing Tools Open Source GIS Routing Tools Figure 5 – Multi Criteria Ranking Support

16. 13 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Defining Ranking Criteria & Its Measures Those criteria considered in this research – i.Routing Output/Drive Test Error margin i.Capacity For Multiple Routing ii.Base-Map content/completeness iii.Support For Traffic Input iv.Routing Platform v.Alternative Route Option The measure of these criteria (table 4) were based on established and referenced techniques (Haklay method). Expert judgement was used to determine % coverage in each cluster. Table 4 - Definition of Ranking Criteria Open Source GIS Routing Tools Open Source GIS Routing Tools

17. 14 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Threshold Definition and Weighing The thresholds for each of the criteria were defined and ordinal weights assigned (table 5). Threshold for base map completeness was derived from average consensus option expressed in percentage for both five sample clusters in rural and urban. The threshold values for rural and urban clusters were determined as 50% and 70% respectively by consensus. Table 5 - Normalized Thresholds Open Source GIS Routing Tools Open Source GIS Routing Tools Hence the average (60%) of the two was used for assigning weight of 1 for greater than 60% and 0 for less than.

18. 15 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Research Outputs: Desktop Routing The outcomes derived was contrary to expectation that routing tools using same base map would have same outputs. QRG, GH, OSRM and OsmAnd use same base map yet the routing outputs varied between 467.26 km (min) for QRG to 488.62 km (max) for OSRM (table 6). This represents about ± 21.36km discrepancy which is about 4% of the entire distance coverage. This may be understandable for GME because different base map was used. Table 6 - Desktop Routing Outputs Open Source GIS Routing Tools Open Source GIS Routing Tools

19. 16 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Research Outputs: Desktop Routing (cont’d) The choice of different routes used by the routing tools during the desktop routing exercise (figure 6) has been responsible for the discrepancy noted in the desktop routing outputs (table 6). This choice was largely influenced by 1.Geo-positioning accuracy of the routing tools, 2.Base map quality in terms of content and completeness, 3.In-built routing algorithms Figure 6 – Non uniform Choice of delivery routes during Desktop Routing Estimation Open Source GIS Routing Tools Open Source GIS Routing Tools

20. 17 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Research Outputs: Comparative Error Margin There were discrepancies in the error margin reported between the drive test average and routing outputs (table 7). Factors responsible for these errors are not limited to the use of different routing algorithms (a case for future investigation) but also the base map quality in terms of content (completeness) as well as geo- positioning accuracy of the routing tools. Table 7 – Comparative Discrepancy Outcomes Open Source GIS Routing Tools Open Source GIS Routing Tools

21. 18 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Scoring/Ranking Outcome In view of this outcome (table 8) with reference to normalized threshold table (5), scoring was derived based on frequency of 1 occurrence while the highest frequency of 4 was ranked 1 st and least score of 1 was ranked 4 th. Based on the overall criteria, QRG emerged 1 st ranked because it’s the only one that supported desktop routing platform and traffic modelling while OsmAnd was last ranked. Both GME and GH were ranked 2 nd (table 9). Table 8 - Normalization Outcome Table 9- Ranking Outcome Open Source GIS Routing Tools Open Source GIS Routing Tools

22. 19 - - A pragmatic support for routine immunization and health care delivery in Northern Nigeria. Scheduling delivery and routing within Kano metropolis was largely constrained by lack of traffic details, hence the routing outputs which considered speed limit as an input for travel time estimation was not consistent with travel time output derived from drive test survey. Thus it is anticipated that future research would consider investigating metropolitan traffic. Speed limit consideration is still valid for interstate and rural areas. QRG is at the moment constrained by inability to handle batch routing. All the routing tools considered are equally unable to do batch routing. Hence it is expected that future research would focus of developing batch routing component and to integrate other features such as alternative route options. RESEARCH FINDINGS, CONCLUSION AND RECOMMENDATIONS It is not surprising to find significant variation in the routing outputs of tools using different base maps (OSM and google) because different choice of routes are chosen. But to discover a significant variation in routing outputs of routing tools using same OSM base map is worrisome. It is an indication that there is an inconsistency in the routing algorithm used. The outcome of drive test clearly shown that GH tool has a better routing algorithm with lowest cumulative error margin. Future research is hereby encouraged to investigate how road graph plugin of QRG would integrate GH routing algorithm for better performance. It is important to emphasize that the GME better drive test performance over QRG was just because the selected delivery routes were largely within urban area where GME base map content is relatively good. It is thus recommended that outstanding GME features such as alternative routing option should be considered in QRG integration. Open Source GIS Routing Tools Open Source GIS Routing Tools

23. THANKS FOR COMING ANY QUESTIONS? An Evaluation of Open Source GIS Routing Tools in Direct Vaccine Delivery in Kano State, Northern Nigeria / 18 September 2015