Validating the Prognosis DDS model for the Inland Empire Robert E. FroeseAndrew P. Robinson School of Forest Resources Etc.Department of Forest Resources Michigan Technological UniversityUniversity of Idaho Forest Biometrics Lab

What is “validation”? Evaluation Verification Validation Corroboration Qualification Model testing is optimally the responsibility of the model user, who is in the best position to clearly state goals and objectives (Brand and Holdaway 1983; Robinson and Ek 2000) assessment would be simpler if model developers report extensive performance information, rather than leave it to model users to generate for themselves (Brand and Holdaway 1983) TESTING X

Purposes for model testing Caswell (1976): modelling has two core purposes, prediction and understanding, which can be distinguished by interest in truth or reality a simplistic example: –the statement “Y varies in the same direction as X” is embodied in the model Y = b 0 + b 1 ·X (b 1 >0); a test of the accuracy of the model may show it to be a poor predictor, while a test of the statement may corroborate it. In other words, –does a model user care if the internal structures are truthful, as long as the model makes accurate predictions? –does the scientist care if the model makes accurate predictions, as long as the model is useful for testing hypotheses about the underlying system? objectives must be clear in design, application and evaluation!

A little bit about Prognosis growth engine for the Forest Vegetation Simulator in the inland empire collection of models - and a model framework –Increment –Mortality –Regeneration –Scheduling and many others

The Inland Empire

What’s inside Prognosis? trees are grown in three dimensions –basal area growth –height growth –crown ratio divided into two classes for modelling –in northern Idaho, large is > 7.62 cm DBH, or > 3 m tall; the others are small

What’s inside Prognosis? Probabilistic – keep track of sampling fraction Stochastic – record tripling and random deviates No Site Index – uses habitat type and other site descriptors 1/300 acre

Wykoff’s 1990 basal area growth model The DDS model is the key driver for increment … because predictions are used directly or indirectly as predictors in other model sub- components DDS = DBH 2 t+10 - DBH 2 t but actually.. DDS = DBH 2 t - DBH 2 t-10 BAG = (π/4)·(DBH 2 t - DBH 2 t-10 ) DG = (DBH 2 + DDS) DBH ln(DDS) = f (SIZE + SITE + COMPETITION)

the 1990 DDS model formulation b i – coefficients estimated by ordinary least squares, of which: –b 0 depends on habitat type and nearest National Forest –b 2 depends on nearest National Forest –b 12 depends on habitat type

Objectives 1.Produce performance information bias Precision 2.Provoke and guide future development examine performance against individual predictors 3.Examine the model as a scientific statement does the model behave the way it should based on biological principles

Not Objectives Traditional hypothesis test for model bias –e.g., H o is of no difference and H a is of a difference Arbitrarily small differences are detectable Statistical significance is not practical significance An alternative: see Andrew Robinson’s talk tomorrow!

FIA Data data from the USDA Forest Service - Forest Inventory and Analysis Program (FIA) geographically extensive –(now) one National design, all forest land ownerships –but… plot locations are strictly confidential unbiased sampling design –systematic random sample –one field location per 2,400 hectares estimates of between and within-stand variability –cluster of 5 to 10 (old design) or 4 (new design) plots retrospective measurements of growth

Methodology 1.backdate FIA stand conditions following Wykoff’s (1990) rationale 1.Find diameter at t-10 for all trees, to calculate competition variables 2.Find height at t-10 for growth sample trees 2.generate predictions using the 1990 DDS model 3.For growth sample trees 1.Calculate a basal area increment prediction residual 2.Estimate volume increment and volume increment prediction residual

Results, overall 40,979 trees over 2,632 FIA field locations for Basal Area Increment –mean increment is cm 2 dec -1 –mean bias is 13.2 cm 2 dec -1 or 11.8% underprediction –bias SD is 76.8 cm 2 dec -1 or CV is 651% for Volume Increment –mean increment is 44.5 m 3 ha -1 dec -1 –mean bias is 1.2 m 3 ha -1 dec -1 or 2.6% underprediction –bias SD is 11.6 m 3 ha -1 dec -1 or CV is 966% this means –2,632 locations x 2,400 ha·location -1 x 1.2 m 3 ha -1 dec -1 = more than 7.6 million m 3 dec -1 underprediction

for Basal Area Increment

By nearest National Forest

for Volume Increment

Trends with predictors

Discussion FIA data –comparable in size and geographic extent –can’t do (precise) spatial analyses BIAS –Practically, 7.5 million m 3 is meaningful –relative to SD, perhaps not meaningful extrapolation –space, time consequences –management for timber –management for non-timber LOC is a problem

Conclusions The 1990 DDS model is not a particularly accurate predictor of forest growth, but it is relatively robust as a theoretical statement under substantial extrapolation in time and space Model users may wish to apply a multiplier to the diameter increment model subcomponent Model development in the future should re-evaluate LOC and look for alternatives

Acknowledgements Funding provided by the USDA Forest Service RMRS RJVA and the University of Idaho Forest Biometrics Lab. This research was completed entirely using open source software. Special thanks to: My major Professor, Dr. Andrew Robinson Bill Wykoff, Moscow Forest Sciences Lab Sharon Woudenberg and John Nelson, FIA Ogden, Utah.