1. Jon Brodziak1, Teresa A’mar2 , Matthew Supernaw2,
Designing a General MSE Framework for a Movement-Based Metapopulation Assessment System
The object-oriented design of MAS, a metapopulation assessment system is described.
Matthew is NMFS scientific programmer. Clay is stock assessment expert and Division
Chief in the SEFSC. I am a math & stats guy from Honolulu.
Jon Brodziak1, Teresa A’mar2 , Matthew Supernaw2,
1Pacific Islands Fisheries Science Center 2NMFS Office of Science and Technology

2. Design Feature: Automatic Differentiation with Analytics Template Library
First Order Automatic Differentiation
ADMB, CASAL, TMB, ATL
Second Order Exact Automatic Differentiation
Exact Hessian and Covariance with ATL
Third Order Exact Automatic Differentiation
Exact Third Order Partials for Random Effects Using Integrated Nested Laplace Approximation with TMB or ATL

3. {M} Design Feature: Parallelization For
Next Generation Stock Assessment Models
Nonlinear Optimization
ADMB, CASAL, TMB, ATL
Parallelization via ATL
Increase speed through parallelization
Application to model ensembles
MAS will need a fast numerical optimization routine
to compute likelihoods for high-dimensional models.
Efficient concurrency through parallelization of model analyses
is a critical need.
MAS is designed to work with ensembles of models.
Beginning with the end in mind, MAS uses model input/output
objects to streamline model processing M1
Fit
Summary
Project
{M} M2
Fit
Summary
Project M3
Fit
Summary
Project

4. Next Generation Stock Assessment Models
Metapopulation Assessment System (MAS)
Object-oriented design
Spatially explicit
The model is the fundamental unit with four model analysis layers:
Model construction
Model selection
Model forecasting and simulation testing
Management strategy evaluation
Our vision of MAS as a next generation modeling approach includes the
following features: object-oriented design and spatially explicit dynamics.
The model is the fundamental unit with three layers of analysis.
The vision is to create a set of function libraries that can be dynamically
linked with a GUI interface to construct a candidate model set, to select
a set of credible models or model, and to forecast future stock conditions
and benefit streams under uncertainty.
A population means an intraspecific group of randomly mating individuals whose
demographic and genetic trajectories are mostly independent of other such groups

6. Model Class Structure Observation Class Analysis Class
Environment Class c
Population Class
Movement
Population
Structure
MAS has a high level model class structure
with four primary classes.
Habitat Location

7. Spatial Dynamics

8. Object-Oriented Paradigm

9. Operating & Assessment Model Object Oriented Paradigm
Submodel Polymorphism
Case Array Specification
Population
Area Structure
Population Recruitiment
Population Size Composition
Data Weighting
Population Growth
MAS will need a fast numerical optimization routine
to compute likelihoods for high-dimensional models.
Efficient concurrency through parallelization of model analyses
is a critical need.
MAS is designed to work with ensembles of models.
Beginning with the end in mind, MAS uses model input/output
objects to streamline model processing
“All models are wrong, some are useful” – G.P. Box

10. Operating & Assessment Model OOP
Submodel Polymorphism
Structural Uncertainty Grid Construction
From Base Case Model
Area Structure
Recruitment
Growth
Base Case
Model =
Abundance Index Error Distribution
MAS will need a fast numerical optimization routine
to compute likelihoods for high-dimensional models.
Efficient concurrency through parallelization of model analyses
is a critical need.
MAS is designed to work with ensembles of models.
Beginning with the end in mind, MAS uses model input/output
objects to streamline model processing
Structural Uncertainty Grid =
Size Composition Data Weighting

11. Improve Predictive Accuracy Via Multi-Model Inference
Bias
Variance
Few
Many
Number of Model Parameters
Expected
Log Predictive
Density

12. Operating & Assessment Model Object Oriented Paradigm
Class Inheritance
Operating Model Class Specifications
MAS will need a fast numerical optimization routine
to compute likelihoods for high-dimensional models.
Efficient concurrency through parallelization of model analyses
is a critical need.
MAS is designed to work with ensembles of models.
Beginning with the end in mind, MAS uses model input/output
objects to streamline model processing
Assessment Model Class Specifications

13. Do Simulation Experiment
Operating Models
Management Strategies
Implementation
Model
Set Harvest
Rate
Model
Management
Measures
Population
Dynamics and Fishery
Models
Simulated
Data
Data
Generation
Model
Estimation
Model
Do Simulation Experiment
Performance
Metrics

14. Operating Model Specifications
Extra Variation
Process Error with Finite Mixture Distributions
Process Error
MAS will need a fast numerical optimization routine
to compute likelihoods for high-dimensional models.
Efficient concurrency through parallelization of model analyses
is a critical need.
MAS is designed to work with ensembles of models.
Beginning with the end in mind, MAS uses model input/output
objects to streamline model processing

15. Management Strategy Evaluation
Set Management Objectives
Choose Uncertainties
Yes
Refine
Analyses ?
Do Simulation Experiment
Construct Operating Models No
Stop
Construct Management Strategies
Set Operating Model Parameters

16. Some Major Sources of Uncertainty For Stock Assessment and Management Advice Processes
Model Uncertainty/Structural Complexity
Estimation Error
Sampling/Observation Error
Natural Variability/Process Error
Implementation Uncertainty
Inadequate Communication
“To know one’s ignorance
Is the best part of knowledge”
~ Lao Tzu, Tao-te Ching, no. 71

17. Assessment Model Ensembles To Account For Structural Uncertainty
Knowledge
Base

18. Forecast Model Ensembles To Account For Structural Uncertainty

19. Measuring Assessment and Forecast Model Performance Under Management Strategies

20. Some Binding Limitations of Multimodel Inference for MSE
Lack of resources
Lack of multimodel assessment examples, see
An Applied Framework for Incorporating Multiple Sources of Uncertainty in Fisheries Stock Assessments. Finlay Scott, Ernesto Jardim, Colin P. Millar PLOS ONE. or better yet
Stock Assessment of Bigeye Tuna in the Western and Central Pacific Ocean. Sam McKechnie, Graham Pilling, John Hamption WCPFC SC13, SA-WP-05,149 p.
Available at:
Inherent complexity of assessment and forecast combinations and management strategies
Inertia to change status quo

21. Some Open Opportunities
Existing resources can be refocused
Many potential good examples can be developed from existing assessment models
Generalizing existing software platforms
Pushing the envelope instead of turning the crank
“The results have been virtually unanimous:
combining multiple forecasts leads to increased forecast accuracy”
~ Clemens (1989. International Journal of Forecasting, 5:
“And in a multitude of counselors there is safety”
~ Proverbs 24:6B

22. Concluding Thoughts Use OOP for AM & MSE Modeling Software
Implement Documentation and Testing Protocols
Design for Maintenance and Extensibility

23. Jon Brodziak, Teresa A’mar
Thanks and Mahalo !
2017 AFS MSE Symposium
Jon Brodziak, Teresa A’mar
& Matthew Supernaw
NOAA Fisheries