dtk-tools Benoit Raybaud, Research Software Manager

dtk-tools Experiments and analysis IDM Weather and Demographics service Models On premise cluster Cloud environments dtk-tools To carry out experiments and analysis we rely on models. Those models need to be parametrized and files usually coming from the local machine or some sort of services They also need to run somewhere: Either the local machine or some high performance computing services Dtk-tools is the suite of tools allowing to put it all together Local files Local computer

Overview Provide a shared way for the different teams to create and execute workflows involving experiments, analysis, and calibrations Easy to reproduce, share and reuse those workflows Abstracts a lot of technical minutia Lower the technical expertise requirement to leverage available resources Accelerate the creation of large experiments Provide a central repository for utilities Before this framework was created each teams had their own way of running workflows and using the IDM ecosystem. By providing a common way to leverage the available resources we enabled users to easily share reuse and reproduce the workflows and to abstract a lot of technical minutia. This led to lowering the technical expertise required and allowed our users to focus more on the science and less on the software. Now that dtk-tools is widely adopted it also provides a centralized repository for all sorts of utilities.

Technical details Written in Python 3.6 Free, open source, and easily extensible Supports EMOD and CMS Disease specific packages Malaria HIV TB Typhoid Disease specific packages : The different diseases have custom reports for each of them and analyzers to consume them

Main features – Input files EMOD needs input files to represent experiments’ environment Heterogeneous sources Local files IDM Weather and Demographics Input service National services (NOAA, WorldPop, etc.) Not all models need weather

IDM Weather and Demographics Service Input files workflow IDM Weather and Demographics Service Climate for 2014 – 2016 Resolution: 2.5’ Custom climate Local files Custom temperature for a given location A usual workflow would be to query the IDM service for the climate in a given place, for a given timestamp. You can query the service directly within the tools and retrieve the set of files. Then Let’s imagine I want to override the temperature for a given location with a CSV file that I have created locally. Well, the tools let you easily use both the base weather and the custom temperature file to create a custom weather. We also provide utilities to transform this custom weather into model readable files. You can then use those files with your simulations. It is interesting to note that we also provide utilities to go the other way and make model files readable. Model consumable set of files

Main features – Model parameters Models need parameters and scenarios EMOD Model needs parameters and interventions definition files The tools provide Starting points with default files for different types of simulations Convenient ways of editing model parameters Shortcut functions allowing to accelerate the creation of interventions

Main features – Asset collections When running simulations, collects all the necessary files Packaging handled transparently by the tools Easy to share, reproduce, and reuse Namawala input files EMOD 2.13 executable New collection Local custom files Creation of a new asset collection from different collections

Main features – Sweeps Convenient and easy way of exploring parameters space Accelerate the creation and tagging of simulations Coverage Multiplier 20% 0.5 1 2 … 75% 5 Base configuration files Simulation Simulation Simulation Sweep Definition Coverage: 20% to 75% (5% step) Rainfall multiplier: 0.5, 1, 2, 5 12*4 = 48 Simulation Simulation Creation of a sweep

Main features – Templating Templating system to modify specific blocks within the model inputs Build complicated scenarios easily Base campaign Scenario definition 10% A 0.5 20% 1 30% 2 B 5 Simulation Intervention 1 Simulation Intervention 2 Simulation Simulation Intervention 3 Simulation Simulation Creation of an experiment based on templates

Main features – Analyzers Simulations can provide a lot of results but usually only a subset of those results are relevant Analyzers are extracting needed per simulation data and then combining results to support investigations Simulations outputs Analysis Simulations can provide a lot of results but usually only a subset of those results are relevant. Analyzers are extracting the relevant data from each simulations and then combine those together to support your investigations. This analyzer process works as follow: In one hand, we have a set of simulations outputs that can come from multiple experiments In the other hand we have our analysis that can include one or more analyzers (here 3) Each analyzers will automatically select the data it needs and produce the output. Analyzers can span a wide range of outputs, from charts to CSV files or heatmaps but they are flexible enough to basically be able to generate anything that Python can handle. Also those analyzers are not tied to a specific experiment or users and can be shared. Another major advantage is that you are not required to know where those outputs files are located, the tools are taking care of all the technical details for you to let you focus on how you want to process the data. Analysis process

Main features – Calibration Enables exploration of a multidimensional parameter space Fit the model outputs based on reference data Several algorithms available to optimize search in the parameter space Set model parameters that cannot be measured Reasonable parameter set when data is sparse

Calibration – Overview Study Site Base simulation Reference data Model Run the scenarios Produce outputs Comparison Compare with reference Assign a likelihood New parameters Likelihoods Search state Monitoring Next point algorithm Define next best set of parameters to try

How to get the tools? GitHub repository: https://github.com/InstituteforDiseaseModeling/dtk-tools Private repository Contact us: support@idmod.org

Thank you !