Modelling in Physical Geography Martin Mergili, University of Vienna Model validation Martin Mergili, University of Vienna

Model validation It is easy with GIS to impress people with nice colourful maps produced with computer models However, we have learned that model parameters are uncertain and also the process understanding is often limited These uncertainties reflect themselves in the model results Model results have to be evaluated against observations to test their validity Limitations Inventory Strategies Instructions 2 2

Landslide inventories In the field of landslide modelling, landslide inventories are most useful for model validation Landslide inventories are freely available for some countries and regions Good landslide inventories are a lot of work to prepare Release and deposition areas should be represented by separate polygons The availability of a landslide inventory is a key requirement for landslide modelling efforts Limitations Inventory Strategies Instructions 3 3

Model development and validation areas If the landslide inventory is used for model development (statistical models) it is always necessary to clearly separate between the model development area and the model validation area In such a case, many model runs with different development and validation areas may be performed in order to achieve robust results This separation is not necessary for physically- based models so that the entire study area may be used for model validation. Limitations Inventory Strategies Instructions 4 4

Simple overlay This only works with binary results Observed landslide area TN = True negative prediction TP = True positive prediction FP = False positive prediction FN = False negative prediction Modelled landslide area (FOS < 1) FP True positive rate rTP = TP / (TP + FN) False positive rate rFP = FP / (FP + TN) TN FN TP FN This only works with binary results FP Limitations Inventory Strategies Instructions 5 5

AUCROC in range 0–1 is indicator for prediction quality ROC Plot 1 Observed landslide area Modelled landslide probability 0.0 rTP AUCROC = 0.5 (random prediction) 0.3 AUCROC 0.6 0.0 0.6 rFP 1 0.3 AUCROC in range 0–1 is indicator for prediction quality True positive rate rTP = TP / (TP + FN) False positive rate rFP = FP / (FP + TN) Limitations Inventory Strategies Instructions 6 6

Instructions We will use an ROC Plot to evaluate our computed slope failure probability map against the landslide inventory for the Sant‘Anna test area. This ROC Plot will be created with the statistical software R. The corresponding R script will be provided. This script will be called directly from the Python script so that the validation procedure is directly included in the work flow of our model application. We will run our model application with varying ranges of the input parameters in order to explore how sensitive the prediction quality reacts. Limitations Inventory Strategies Instructions 7 7

Instructions Python 2.7 documentation: https://docs.python.org/2.7/ ArcPy documentation: https://desktop.arcgis.com/en/desktop/latest/analyze/arcpy/a-quick-tour-of-arcpy.htm R documentation: https://www.r-project.org/ Limitations Inventory Strategies Instructions 8 8

Have fun! martin.mergili@univie.ac.at http://www.mergili.at 9 9