NPS Conference 2006 # 1 WE PROBABLY COULD HAVE MORE FUN TALKING ABOUT THESE TRAFFIC STOPPERS

NPS Conference 2006 # 2 WHO CLEARLY HAVE THE RIGHT OF WAY! BUT…

NPS Conference 2006 # 3 DESIGNING SURVEYS OVER TIME (PANEL SURVEYS) VARIANCE, POWER and RELATED TOPICS N. Scott Urquhart Senior Research Scientist Department of Statistics Colorado State University Fort Collins, CO

NPS Conference 2006 # 4 BRIEF COMMENTS ON MONITORING Monitoring is a long-term endeavor for most (living) natural resources Monitoring is a long-term endeavor for most (living) natural resources Think of it as a legacy for your grandchildren Many of you have tried to use solid data someone else gathered, but documented poorly. Many of you have tried to use solid data someone else gathered, but documented poorly. Much good data dies when the person who gathered it dies or retires! Monitoring data requires three things to retain its value: Monitoring data requires three things to retain its value: Metadata Storage in a retrievable, maintained, data system Backed up – safe from fires, floods, or earthquakes

NPS Conference 2006 # 5 SHORT OUTLINE OF THE REST OF THIS TALK Inference Perspectives & monitoring Inference Perspectives & monitoring Trend, Variance Structures, and Power to Detect Trend Trend, Variance Structures, and Power to Detect Trend Panels and Panel Structures Panels and Panel Structures Putting it all together = Power Curves {and standard errors of estimated current status} Putting it all together = Power Curves {and standard errors of estimated current status} Status Estimates from Data Over Years Status Estimates from Data Over Years

NPS Conference 2006 # 6 INFERENCE PERSPECTIVES Design Based Design Based Inferences rest on the probability structure incorporated in the sampling plan Completely defensible; very minimal assumptions Limiting relative to using auxiliary information Model Assisted Model Assisted Uses models to complement underlying sampling structure Has opportunities for use of auxiliary information Model Based (eg: spatial statistics) Model Based (eg: spatial statistics) Ignores sampling plan Defensibility lies in defense of model

NPS Conference 2006 # 7 APPROACH OF THIS PRESENTATION Use tools from the arena of Use tools from the arena of Model-assisted and Model-based analyses To study the performance of To study the performance of Design based & Model-assisted analyses WHY? WHY? Without models, performance evaluations need simulation (Steve Garmans topic) Before substantial data have been gathered Before substantial data have been gathered Minimal basis for values to enter into simulation studies

NPS Conference 2006 # 8 STATUS & TRENDS OVER TIME IN ECOLOGICAL RESOURCES OF A REGION MAJOR POINTS Regional trend site trend Regional trend site trend Detection of trend requires substantial elapsed time Detection of trend requires substantial elapsed time Regional OR intensive site Almost all indicators have substantial patterns in their variability Almost all indicators have substantial patterns in their variability Design to capitalize on this; dont fight it. Minimize effect of site variability with planned revisits – specific plans will be illustrated Minimize effect of site variability with planned revisits – specific plans will be illustrated Design tradeoffs: TREND vs STATUS Design tradeoffs: TREND vs STATUS

NPS Conference 2006 # 9 REGIONAL TREND SITE TREND The predominant theme of ecology: The predominant theme of ecology: Ecological processes How does a specific kind of ecosystem function Energy flows Food webs Nutrient cycling Most studies of such functions must be Temporally intensive –What material goes from where to where? Consequently spatially restrictive In this situation: Temporal trend = site trend

NPS Conference 2006 # 10 REGIONAL TREND SITE TREND ( - CONTINUED) The predominant theme of ecology versus The predominant theme of ecology versus A Substantial (any) Agency Focus: A Substantial (any) Agency Focus: All of an ecological resource In an area or region Across all of the variability present there For Example, National Park Service For Example, National Park Service All riparian areas in Olympic National Park All riparian areas in National Parks in the coastal Northwest

NPS Conference 2006 # 11 TREND ACROSS TIME - What is it? Any response which changes across time in a generally Any response which changes across time in a generally Increasing or Decreasing Manner shows trend Manner shows trend Monotonic change is not essential. If trend of this sort is present, it WILL BE detectable as linear trend. If trend of this sort is present, it WILL BE detectable as linear trend. NOT This does NOT mean trend must be linear (examples follow) Any specified form is detectable Time = years, here

NPS Conference 2006 # 12 TREND ACROSS TIME - What is it? (continued)

NPS Conference 2006 # 13

NPS Conference 2006 # 14 TREND DETECTION REQUIRES SUBSTANTIAL ELAPSED TIME IT IS NEARLY IMPOSSIBLE TO DETECT TREND IN LESS THAN FIVE YEARS. WHY? IT IS NEARLY IMPOSSIBLE TO DETECT TREND IN LESS THAN FIVE YEARS. WHY?

NPS Conference 2006 # 15 POPULATION VARIANCE: POPULATION VARIANCE: YEAR VARIANCE: YEAR VARIANCE: RESIDUAL VARIANCE: RESIDUAL VARIANCE: VARIANCE HAS A LOT OF STUCTURE IMPORTANT COMPONENTS OF VARIANCE

NPS Conference 2006 # 16 HOW SHOULD YOU ESTIMATE VARIANCE? Alternatives: Alternatives: Designed-based Horwitz-Thompson – extremely variable = dont use Local Neighborhood Variance Estimator (NBH) – Stevens – Gives estimate of variance something like residual component of variance, only – Power nearly impossible to evaluate in this context Model-assisted Linear models – Urquhart & Courbois (& Williams, now) Gives estimates of site, year and residual variances Which should you use?

NPS Conference 2006 # 17 We really arent sure, but this topic is under active investigation We really arent sure, but this topic is under active investigation Don Stevens and I are good friends, so this topic isnt a professional conflict. Don Stevens and I are good friends, so this topic isnt a professional conflict. We both want to know the answer! Results by Courbois (JABES, 2004) suggest this: Results by Courbois (JABES, 2004) suggest this: Unless response values and inclusion probabilities ( p ) are highly correlated, they ( p ) can be ignored. If this stands up, as I expect it to, practically, linear model estimates of components of variance will be fine. Answer expected by summer. HOW SHOULD YOU ESTIMATE VARIANCE? (Continued)

NPS Conference 2006 # 18 POPULATION VARIANCE: POPULATION VARIANCE: Variation among values of an indicator (response) across all sites in a park or group of related parks, that is, across a population or subpopulation of sites IMPORTANT COMPONENTS OF VARIANCE ( - CONTINUED)

NPS Conference 2006 # 19 YEAR VARIANCE: YEAR VARIANCE: ALL Concordant variation among values of an indicator (response) across years for ALL sites in a regional population or subpopulation NOT variation in an indicator across years at a single site Detrended remainder, if trend is present Effectively the deviation away from the trend line (or other curve) IMPORTANT COMPONENTS OF VARIANCE ( - CONTINUED II)

NPS Conference 2006 # 20 Residual component of variance Residual component of variance Has several contributors Year*Site interaction This contains most of what ecologists would call year to year variation, i.e. the site specific part Index variation Measurement error Crew-to-crew variation (minimize with well documented protocols and training) Local spatial = protocol variation Short term temporal variation IMPORTANT COMPONENTS OF VARIANCE ( - CONTINUED - III)

NPS Conference 2006 # 21 SOURCE OF DATA FOR ESTIMATES OF COMPONENTS OF VARIANCE EMAP Surface Waters: Northeast Lakes Pilot EMAP Surface Waters: Northeast Lakes Pilot About 450 observations About 450 observations Over four years Including about 350 distinct lakes Design allowed estimation of several residual components Lakes illustrate what is generally referred to in this presentation as sites. Lakes illustrate what is generally referred to in this presentation as sites. Similar patterns appear in other data sets Both aquatic and terrestrial

NPS Conference 2006 # 22

NPS Conference 2006 # 23 ALL VARIABILITY IS OF INTEREST The site component of variance is one of the major descriptors of the regional population The site component of variance is one of the major descriptors of the regional population The year component of variance often is small, too small to estimate. It is a major enemy for detecting trend over time. The year component of variance often is small, too small to estimate. It is a major enemy for detecting trend over time. If it has even a moderate size, sample size reverts to the number of years. In this case, the number of visits and/or number of sites has no practical effect.

NPS Conference 2006 # 24 ALL VARIABILITY IS OF INTEREST ( - CONTINUED) Residual variance characterizes the inherent variation in the response or indicator. Residual variance characterizes the inherent variation in the response or indicator. But some of its subcomponents may contain useful management information But some of its subcomponents may contain useful management information CREW EFFECTS ===> training VISIT EFFECTS ===> need to reexamine definition of index (time) window or evaluation protocol MEASUREMENT ERROR ===> work on laboratory/measurement problems

NPS Conference 2006 # 25 DESIGN TRADE-OFFS: TREND vs STATUS How do we detect trend in spite of all of this variation? How do we detect trend in spite of all of this variation? Recall two old statistical friends. Recall two old statistical friends. Variance of a mean, and Blocking

NPS Conference 2006 # 26 DESIGN TRADE-OFFS: TREND vs STATUS ( - CONTINUED) VARIANCE OF A MEAN: VARIANCE OF A MEAN: Where m members of the associated population have been randomly selected and their response values averaged. Here the mean is a regional average slope, so " 2 " refers to the variance of an estimated slope ---

NPS Conference 2006 # 27 DESIGN TRADE-OFFS: TREND vs STATUS ( - CONTINUED - II) Consequently Consequently Becomes Becomes Note that the regional averaging of slopes has the same effect as continuing to monitor at one site for a much longer time period. Note that the regional averaging of slopes has the same effect as continuing to monitor at one site for a much longer time period.

NPS Conference 2006 # 28 DESIGN TRADE-OFFS: TREND vs STATUS ( - CONTINUED - III) Now, 2, in total, frequently is large. Now, 2, in total, frequently is large. If we take one regional sample of sites at one time, and another at a subsequent time, the site component of variance is included in 2. If we take one regional sample of sites at one time, and another at a subsequent time, the site component of variance is included in 2. Enter the concept of blocking, familiar from experimental design. Enter the concept of blocking, familiar from experimental design. Regard a site like a block Periodically revisit a site The site component of variance vanishes from the variance of a slope.

NPS Conference 2006 # 29 PANEL DESIGNS Question: What kind of temporal design should you use for National Parks? Question: What kind of temporal design should you use for National Parks? A Panel is a Set of Sites which have the same Revisit Schedule A Panel is a Set of Sites which have the same Revisit Schedule Each panel ordinarily should have as good a spatial coverage as possible (GRTS) You have many usable and defensible temporal designs You have many usable and defensible temporal designs Choose one which fits your needs and resources Evaluation tools are available, and demonstrated here

NPS Conference 2006 # 30 A SINGLE PANEL Conventional for trend detection Conventional for trend detection Not very good for status Not very good for status (Call this design 1) (Call this design 1) NUM OBS YEAR XXXXXXXXXXXXXXXXXXXXX

NPS Conference 2006 # 31 AN AUGMENTED PANEL PLAN (Design 2) NUMOBSYEARS … 5XXXXXXXXXXXXXXXXXXX… Add sites to the above annual revisit plane, as Add sites to the above annual revisit plane, as

NPS Conference 2006 # 32 AN AUGMENTED PANEL PLAN (Design 2) NUMOBSYEARS … 5XXXXXXXXXXXXXXXXXXX… 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X 6X ……

NPS Conference 2006 # 33 A POSSIBLE NPS PANEL PLAN (Design 3) NUMOBSYEARS … 5XX 5X 5X 5X

NPS Conference 2006 # 34 A POSSIBLE NPS PANEL PLAN (Design 3) NUMOBSYEARS … 5XX 5X 5X 5X 4XXXXX 1 XXXXX 1 XXXXX 1 XXXX…

NPS Conference 2006 # 35 A POSSIBLE NPS PANEL PLAN (Design 3) NUMOBSYEARS … 5XX 5X 5X 5X 4XXXXX 1 XXXXX 1 XXXXX 1 XXXX… 2XXXX 1XXXX 1XXXX 1XX… 2XX 1XX 1XX 1XX 2XX ……

NPS Conference 2006 # 36 WHY LOOK AT POWER? Power provides a tool for comparing various designs Power provides a tool for comparing various designs Looking at it does not imply that we have to conduct tests of hypotheses Looking at it does not imply that we have to conduct tests of hypotheses The computations displayed here use The computations displayed here use A components of variance statistical model To evaluate the underlying variances of estimated slopes And the temporal design being considered These computations are much more complex, and suitable to your problems, than any of the web-available tools Use of those tools here would commit sins of pseudoreplication!

NPS Conference 2006 # 37 POWER FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {large is good}

NPS Conference 2006 # 38 POWER FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {large is good}

NPS Conference 2006 # 39 POWER FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {large is good}

NPS Conference 2006 # 40 POWER FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {large is good}

NPS Conference 2006 # 41 STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {small is good}

NPS Conference 2006 # 42 STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {small is good}

NPS Conference 2006 # 43 STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {small is good}

NPS Conference 2006 # 44 STANDARD ERROR OF STATUS FOR DESIGNS 1 – 3, & 2A WITH PARAMETER CONDITIONS BILLY GAVE {small is good}

NPS Conference 2006 # 45 DESIGN 3 – A PROPOSED NPS TEMPORAL DESIGN?? Defensible design, given restrictions of Defensible design, given restrictions of Field resources = # sites visited per year Need to split field resources across two type of aquatic systems: Streams Wetlands

NPS Conference 2006 # 46 STATUS ESTIMATES FROM DATA OVER YEARS Idea: Idea: Estimate year effects, and Adjust all sites to the latest year Idea very similar to adjusted treatment means in the analysis of covariance For example, if a line approximates the trend, For example, if a line approximates the trend, The array of { Site i + (year now – year initial )slope } Describes current status, accounting for documented trend This approach extends to models other than lines. Linkage of site revisits (=connectedness) is necessary to allow estimation of all differences in year effects Linkage of site revisits (=connectedness) is necessary to allow estimation of all differences in year effects

NPS Conference 2006 # 47 This research is funded by U.S.EPA – Science To Achieve Results (STAR) Program Cooperative Agreement # CR The work reported here today was developed under the STAR Research Assistance Agreement CR awarded by the U.S. Environmental Protection Agency (EPA) to Colorado State University. This presentation has not been formally reviewed by EPA. The views expressed here are solely those of presenter and STARMAP, the Program he represents. EPA does not endorse any products or commercial services mentioned in this presentation. FUNDING ACKNOWLEDGEMENT

NPS Conference 2006 # 48 RELATED INFORMATION ON THE WEB Web-available information from a graduate course in environmental Sampling (ST571) I taught at OSU Web-available information from a graduate course in environmental Sampling (ST571) I taught at OSU Environmental Sampling Anatomy Variable Probability Sampling Cost Effective Resource Allocation Sampling Macroinvertebrates Maps & Grids Spatial Sampling Support Regions Statistical Power - Concepts Power to Detect Trend in Ecological Resources Representative Sampling Statistical Aspects of Taxonomic Richness Sample Size to Estimate Taxonomic Richness Evaluating a Protocol for "Measuring" Physical Habitat Also see: Also see: Also see NPS I&M site, from Port Angeles meeting, 2003 Also see NPS I&M site, from Port Angeles meeting, 2003

NPS Conference 2006 # 49 VISUALIZING LINES AND YEAR EFFECTS N. Scott Urquhart STARMAP Colorado State University Fort Collins, Co

NPS Conference 2006 # 50 WHAT WE SEE – NO LINES & NO DECOMPOSITION

NPS Conference 2006 # 51 A LINE

NPS Conference 2006 # 52 A LINE WITH YEAR EFFECTS

NPS Conference 2006 # 53 A LINE WITH YEAR EFFECTS & ANNUAL RESIDUALS

NPS Conference 2006 # 54 A LINE WITH ONLY ANNUAL RESIDUALS SHOWN

NPS Conference 2006 # 55 A LINE WITH ANNUAL RESIDUALS & END MARKERS

NPS Conference 2006 # 56 A LINE WITH YEAR EFFECTS, JIGGERED RESIDUALS & END MARKERS

NPS Conference 2006 # 57 A REALITY With a single site With a single site The year effect And residual effect Can not be separated But with several sites But with several sites These effects can be separated Following figures show the patterns Following figures show the patterns

NPS Conference 2006 # 58 TWO LINES

NPS Conference 2006 # 59 TWO LINES WITH YEAR EFFECTS

NPS Conference 2006 # 60 TWO LINES WITH YEAR EFFECTS & JIGGERED RESIDUALS

NPS Conference 2006 # 61 TWO LINES WITH YEAR EFFECTS & JIGGERED RESIDUALS – ONLY SOME YEARS

NPS Conference 2006 # 62 WHAT WE SEE – NO LINES & NO DECOMPOSITION