1. Combining Species Occupancy Models and Boosted Regression Trees
Rebecca Hutchinson
Joint work with Tom Dietterich
Oregon State University
International Conference on Computational Sustainability
June 29, 2010

2. Outline Motivation and Background Approach and Algorithms
Boosted Regression Trees (BRTs) have been successful in SDM (Elith 06)
Occupancy models account for uncertain detection (e.g. MacKenzie 06)
Approach and Algorithms
Use BRTs on the links of the occupancy graphical models
Functional Gradient Ascent (Friedman 01)
…within EM?
Preliminary Results
Synthetic data
Future Directions

3. Problem slide?

4. Boosted Regression Trees
Have been useful in SDM
Elith 06, other citations?
Pic?

5. Occupancy Models oi dit Xi Zi Yit Wit t=1,…,T i=1,…,M Covariates of
MacKenzie et al, 2006
Occupancy Models
Covariates of
occupancy (e.g.
elevation, vegetation)
Covariates of
detection (e.g.
time of day, effort)
Probability of occupancy
(function of Xi, a)
Probability of detection
(function of Wit, b) oi
dit Xi Zi
Yit
Wit
t=1,…,T
Visits
i=1,…,M
Sites
True (latent) presence/absence
Zi ~ Bern(oi)
Observed presence/absence
Yit ~ Bern(Zidit)
Assumptions:
population closure across visits
site independence
no misidentifications

6. Objective Functions Marginal Conditional Log-likelihoodoi
dit Xi Zi
Yit
Wit
i=1,…,M
Marginal Conditional Log-likelihood
Expected Joint Log-likelihood

7. Parameterizations Traditional logistic regression
Boosted regression trees
F and G are ensembles of trees

8. Algorithms For either objective function we can do:
1. Gradient ascent
Derivatives w.r.t. a and b in the linear parameterization of o and d
One for each parameter
2. Functional gradient ascent
Derivatives w.r.t F and G in the tree ensemble parameterization of o and d
One for each data point
Friedman 2001

9. More on Functional Gradient Ascent?

10. Regularization and Tuning
L1 or L2
separate weights for penalties on occupancy and detection components in {0, 0.001, 0.01, 0.1, 1}
chosen for highest log-likelihood on validation set
Tuning parameters for tree ensembles
tree depth in {1, 2, 3, 5}
shrinkage (learning rate) in {0.001, 0.01, 0.1}
number of trees/stages of boosting [1,2000]

11. Synthetic Data Generated from the occupancy model
250 train sites, 250 validation sites, 250 test sites
2 visits each
Standard normal covariates
Different true functions F and G
1. linear function of 2 covariates
2. XOR of the sign of 2 covariates
3. highly non-linear with 50 covariates, some irrelevant
4 combinations of objective function and parameterization

12. Evaluation Metrics Log-likelihood of the test set AUC log P(Y|X,W)
predicting Z (not available on real datasets)
predicting Y

13. Dataset #1: Linear Method Tuning parameters Train LL Test LL AUC on Z
AUC on Y
Truth
-218
-214
0.821
0.914 LR
L2, wts (1, 0.1)
-216
0.822
0.792
LR + EM
L2, wts (1, 0.5)
-217
Trees
Depth 2, shrinkage 0.1, 1000 stages
-201
-227
0.802
0.769
Trees + EM
-253
-264
0.766
0.729

14. Dataset #2: XOR Method Tuning parameters Train LL Test LL AUC on Z
AUC on Y
Truth
-170
-178
0.640
0.903 LR
L1, wts (0,1)
-260
-269
0.496
0.520
LR + EM
L1, wts (0.01, 1)
0.497
0.522
Trees
Depth 5, shrinkage 0.1, 4 stages
-217
-248
0.586
0.684
Trees + EM
Depth 5, shrinkage 0.1, 1000 stages
-261
-266
0.599
0.654

15. Dataset #3: Complex Method Tuning parameters Train LL Test LL AUC on Z
AUC on Y
Truth LR
LR + EM
Trees
Trees + EM

16. Future Directions Remove the assumption of no misidentifications
Continue exploring issues on synthetic data
Apply to eBird

17. References
J. Elith et al, “Novel methods improve prediction of species’ distributions from occurrence data.” Ecography, 2006.
J.H. Friedman, “Greedy function approximation: A gradient boosting machine.” The Annals of Statistics, 2001.
D.I. MacKenzie et al, Occupancy Estimation and Modeling: Inferring Patterns and Dynamics of Species Occurrence, 2006.