A COMPARISON OF APPROACHES FOR VERIFYING SOUTHWEST REGIONAL GAP VERTEBRATE-HABITAT DISTRIBUTION MODELS J. Judson Wynne, Charles A. Drost and Kathryn A. Thomas USGS- Southwest Biological Science Center, Colorado Plateau Research Station, Flagstaff, Arizona

Southwest Regional Gap Data products: land cover, stewardship and wildlife-habitat maps 833 wildlife habitat maps Information to land managers, researchers, policy makers, and the general public AZ, CO, NM, NV, UT - Habitat correlates identified via literature reviews

Objectives Compare and contrast the two approaches. Review approach to accuracy assessment used in previous GAP programs. Describe an alternative approach to GAP accuracy assessment. Identify best approach.

Accuracy Assessment Agreement between the expected and observed Utility: - Evaluate model quality - Identify and correct for error - Compare techniques, algorithms, and model developers - Assess relevance in decision making

Without Accuracy Assessment… Models are untested hypotheses Does =+ ? = ? And does

Gap and “Accuracy Assessment” Species list approach - compiled list of species within given management areas - overlaid on predictive distribution maps - “measure of agreement” with omission and commission errors

Sample Size Arizona Accuracy Assessment GAP1 - Source data: USDA FS, NPS, AGFD, TNC 10Mammals 15Birds 10Reptiles 10Amphibians No. of AreasTaxonomic Group From Drost et al. (1999) 1. Intensive, area wide for ≥ one taxon - Criteria 2. Compilation of several surveys with ≥ one taxon 3. Compilation of secondary sources

Expansion Range Issues Require recent data to assess these issues Contraction

False Agreement Incidental/ accidental occurrences

False Agreement Observations not within habitat

Issues of Temporal Scale Time frame reflected in data

Issues of Spatial Scale For all groups, model accuracy <50% for areas <1000ha Accuracy increased as species list area increased Mammals < Birds Reptiles Amphibians P-valueR2R2 No. of Areas Taxonomic Group Birds Arizona Accuracy Assessment GAP 1

Approach: Use of species occurrence data Measure of habitat rather than indirect measure of range More appropriate scale Statistically meaningful accuracy metrics A Higher Standard

- Presence only point data - Grid-based data - Presence/ absence point data Data Sources for Model Verification

Presence Only Data USDA Forest Service Arizona / New Mexico database - Point data - Museum specimen, trapping and other observations - Standardized format

Presence/ Absence Data AGFD Bat Data (Central Arizona) - Systematic sampling protocols - Seven year dataset - Multiple surveys/ year/ site - Point data

Grid Data Arizona Breeding Bird Atlas - Appropriate scale (~1 km 2 grid cell) - Rigorous sampling design - Recently completed - Statewide coverage - Sampling- multiple years/ site - Trained/ experienced observers

Systematic Typically random Data Collection Defined by sample design Variable Reliability Much larger Very small Sample Size Limited to comprehensive Poor Coverage Known for each dataset Generally not addressed Temporal Issues Close to map scale Very coarse Scale Verification DataSpecies List Comparison of Approaches

Conclusions Best approach: Occurrence/ verification data Verification data do exist Best available science Species lists inadequate for accuracy assessment