Validating Wykoff's Model, Take 2: Equivalence tests and spatial analysis in a design- unbiased analytical framework Robert Froese, Ph.D., R.P.F. School of Forest Resources and Environmental Science Michigan Technological University, Houghton MI

This presentation has six parts Introduction Methods Equivalence Trends Relevance Performance The model and the objectives The region, the data and the approach How well does Wykoff’s model predict? Does Wykoff’s model pass a validation test? How does accuracy relate to model and data structures What does it mean for model users and future revisions

This presentation has six parts Introduction Methods Equivalence Trends Relevance Performance The model and the objectives The region, the data and the approach How well does Wykoff’s model predict? Does Wykoff’s model pass a validation test? How does accuracy relate to model and data structures What does it mean for model users and future revisions

This presentation has six parts Introduction Methods Equivalence Trends Relevance Performance The model and the objectives The region, the data and the approach How well does Wykoff’s model predict? Does Wykoff’s model pass a validation test? How does accuracy relate to model and data structures What does it mean for model users and future revisions

This presentation has six parts Introduction Methods Equivalence Trends Relevance Performance The model and the objectives The region, the data and the approach How well does Wykoff’s model predict? Does Wykoff’s model pass a validation test? How does accuracy relate to model and data structures What does it mean for model users and future revisions

This presentation has six parts Introduction Methods Equivalence Trends Relevance Performance The model and the objectives The region, the data and the approach How well does Wykoff’s model predict? Does Wykoff’s model pass a validation test? How does accuracy relate to model and data structures? What does it mean for model users and future revisions

This presentation has six parts Introduction Methods Equivalence Trends Relevance Performance The model and the objectives The region, the data and the approach How well does Wykoff’s model predict? Does Wykoff’s model pass a validation test? How does accuracy relate to model and data structures What does this mean for model users and for future revisions?

Wykoff’s model predicts basal area increment but is used to project diameter Introduction Methods Equivalence Trends Relevance Performance DDS = DBH 2 t+10 - DBH 2 t BAG = (π/4)·(DBH 2 t - DBH 2 t-10 ) DG = (DBH 2 + DDS) DBH ln(DDS) = f (SIZE + SITE + COMP)

Wykoff’s model is a multiple linear regression on the logarithmic scale b i – coefficients estimated by ordinary least squares, of which: –b 0 depends on habitat type and nearest National Forest –b 2 depends on nearest National Forest –b 12 depends on habitat type

This validation is focused on two notions Caswell (1976) introduces two ideas: –does a model user care if the internal structures are truthful, as long as the model makes accurate predictions? –does the scientist care if the model makes accurate predictions, as long as the model is useful for testing hypotheses about the underlying system? Robinson and Froese (2004) question how statistical tests are used for model validation –The usual null hypothesis is of no difference, or that a model is valid, which seems unscientific –Arbitrarily small differences are detectable –A failure to reject may simply imply low power

This study had four objectives and two perspectives The objectives were: to estimate model bias by species across the range of application; to demonstrate a specific validation of Wykoff’s model for diameter increment prediction through a test of equivalence; to identify significant trends between bias and predictor variables, and; to evaluate spatial trends in bias across the geographic area to which Prognosis is usually applied. Two perspectives were taken regarding Wykoff’s model: as a diameter increment model, and; as it contributes to predictions of per hectare volume increment, which is more intuitive or of more interest to many forest managers.

National Forests and geography of the Inland Empire Introduction Methods Equivalence Trends Relevance Performance

1, 2, 4: 3: Inland Empire forests change predictably at various geographic scales

The focus in this study was on geographically extensive individual tree field data Data came from Forest Inventory and Analysis Subject is prediction error Correct for log transform bias + V t - V t-10 =∆V Impute volume increment by backdating

Equivalence tests flip the burden of proof onto the model Select a metric of model performance Nominate an interval of equivalence  –Say 10% of Construct two one-sided confidence intervals of size  If completely contained within the interval, reject the null hypothesis of dissimilarity From Robinson and Froese (2004)

Most FIA plots were variable probability samples and may imply a design bias Introduction Methods Equivalence Trends Relevance Performance

Design unbiased results showed a modest over prediction by Wykoff’s model Extrapolated to the study area, over prediction could be: ~2,400 ha·plot -1 2,632 plots 0.5 m 3 ·ha -1 ·dec -1 = ~3,158,400 m 3 ·dec -1

Equivalence tests are constructed within a regression framework Introduction Methods Equivalence Trends Relevance Performance

Equivalence tests for diameter increment generally fail to validate the model

For stand level volume increment, equivalence tests frequently validate the model

Prediction error is weakly related to most predictor variables Introduction Methods Equivalence Trends Relevance Performance p ≤ 0.01, r 2 ≤ 0.1 in all cases

Diameter prediction error shows a spatial trend irregularly correlated with elevation

Spatial trends in volume prediction error largely mirror those for diameter

In some locations bias appears meaningfully different on and off of National Forest lands Forests that are equivalent have an obvious matrix of public and private land across elevation and geography

Wykoff’s model for prediction Equivalence tests provide an objective methodology for assessing model validity –There is added subjectivity in the selection of I For diameter, a large I would be necessary to validate Wykoff’s model –For most species I = 25% would have to be used –largely because of bias not two-one-sided CI For volume, the model is largely validated –But trends show bias is close to zero for average conditions Overprediction of > 3 mil m 3 dec -1 is not insubstantial Species results differ, and may imply invalid stand dynamics Introduction Methods Equivalence Trends Relevance Performance

Wykoff’s model as a theory Wykoff’s model is surprisingly robust –These tests involve substantial extrapolation in time and space Model improvements should focus on the way climate is represented –LOC as a proxy for regional climate –EL as a global parabolic function –Interactions with other predictors, like DBH 2 –Static proxies or process variables? Other issues remain –Small trees

Summary Wykoff’s model modestly over predicts diameter increment, but the effect on volume is smaller Equivalence tests fail to validate the model for diameter increment, but less often for volume As a theory, the model is surprisingly robust The way climate is represented in the model needs to be addressed ∆D 14% ∆V 2%