1. Modelling in Physical Geography Martin Mergili, University of Vienna
Model validation
Martin Mergili, University of Vienna

2. Model validation
It is easy with GIS to impress people with nice colourful maps produced with computer models
However, we have learned that model parameters are uncertain and also the process understanding is often limited
These uncertainties reflect themselves in the model results
Model results have to be evaluated against observations to test their validity
Limitations
Inventory
Strategies
Instructions 2 2

3. Landslide inventories
In the field of landslide modelling, landslide inventories are most useful for model validation
Landslide inventories are freely available for some countries and regions
Good landslide inventories are a lot of work to prepare
Release and deposition areas should be represented by separate polygons
The availability of a landslide inventory is a key requirement for landslide modelling efforts
Limitations
Inventory
Strategies
Instructions 3 3

4. Model development and validation areas
If the landslide inventory is used for model development (statistical models) it is always necessary to clearly separate between the model development area and the model validation area
In such a case, many model runs with different development and validation areas may be performed in order to achieve robust results
This separation is not necessary for physically- based models so that the entire study area may be used for model validation.
Limitations
Inventory
Strategies
Instructions 4 4

5. Simple overlay This only works with binary results
Observed landslide area
TN = True negative prediction
TP = True positive prediction
FP = False positive prediction
FN = False negative prediction
Modelled landslide area (FOS < 1) FP
True positive rate
rTP = TP / (TP + FN)
False positive rate
rFP = FP / (FP + TN) TN FN TP FN
This only works
with binary results FP
Limitations
Inventory
Strategies
Instructions 5 5

6. AUCROC in range 0–1 is indicator for prediction quality
ROC Plot 1
Observed landslide area
Modelled landslide probability
0.0
rTP
AUCROC = 0.5 (random prediction)
0.3
AUCROC
0.6
0.0
0.6
rFP 1
0.3
AUCROC in range 0–1 is indicator for prediction quality
True positive rate
rTP = TP / (TP + FN)
False positive rate
rFP = FP / (FP + TN)
Limitations
Inventory
Strategies
Instructions 6 6

7. Instructions
We will use an ROC Plot to evaluate our computed slope failure probability map against the landslide inventory for the Sant‘Anna test area.
This ROC Plot will be created with the statistical software R.
The corresponding R script will be provided. This script will be called directly from the Python script so that the validation procedure is directly included in the work flow of our model application.
We will run our model application with varying ranges of the input parameters in order to explore how sensitive the prediction quality reacts.
Limitations
Inventory
Strategies
Instructions 7 7

8. Instructions Python 2.7 documentation: https://docs.python.org/2.7/
ArcPy documentation:
R documentation:
Limitations
Inventory
Strategies
Instructions 8 8

9. Have fun! 9 9