ROMS/TOMS Numerical Algorithms Hernan G. Arango IMCS, Rutgers University New Brunswick, NJ, USA

Developers and Collaborators Hernan G. Arango Alexander F. Shchepetkin W. Paul Budgell Bruce D. Cornuelle Emanuele DiLorenzo Tal Ezer Mark Hadfield Kate Hedstrom Robert Hetland John Klinck Arthur J. Miller Andrew M. Moore Christopher Sherwood Rich Signell John C. Warner John Wilkin Rutgers University UCLA IMR, Norway SIO Princeton University NIWA, New Zealand University of Alaska, ARSC TAMU Old Dominion SIO University of Colorado USGS/WHOI SACLAND USGS/WHOI Rutgers University

(Relief Image from NOAA Animation by Rutgers) Our models are used in oceanographic studies in over 30 countries by: Universities Government Agencies Companies Totaling 280 registered users on six continents

Free-surface, hydrostatic, primitive equation model Generalized, terrain-following vertical coordinates Boundary-fitted, orthogonal curvilinear, horizontal coordinates on an Arakawa C-grid Non-homogeneous time-stepping algorithm Accurate discretization of the baroclinic pressure gradient term High-order advection schemes Continuous, monotonic reconstruction of vertical gradients to maintain high-order accuracy KERNEL ATTRIBUTES

Vertical Terrain-following Coordinates Dubrovnik (Croatia) Vieste (Italy) Longitude Depth (m)

Curvilinear Transformation

Model Grid Configuration Nested Composed

ROMS/TOMS GOVERNING EQUATIONS

Momentum Governing Equations

Tracers Governing Equations

Continuity Equation Vertical Velocity

Deviatoric Transverse Stress Tensor

Parabolic Splines Reconstruction

Dispersive Properties of Advection  /2  /4 3  /4 kxkx 1/2 1 3/2 2 5/2 K(k)  x Parabolic Splines Vs Finite Centered Differences

Barotropic Filters

Barotropic Power-Law Shape Functions

Barotropic-Baroclinic Coupling

ROMS/TOMS: MODULAR DESIGN

Modular, efficient, and portable Fortran code (F90/ F95) C-preprocessing managing Multiple levels of nesting and composed grids Lateral boundary conditions options for closed, periodic, and radiation Arbitrary number of tracers (active and passive) Input and output NetCDF data structure Support for parallel execution on both shared- and distributed -memory architectures CODE DESIGN

Coarse-grained parallelization PARALLEL DESIGN

} } Nx Ny PARALLEL TILE PARTITIONS 8 x 8

Coarse-grained parallelization PARALLEL DESIGN Shared-memory, compiler depend directives MAIN (OpenMP standard) Distributed-memory (MPI; SMS) Optimized for cache-bound computers ZIG-ZAG cycling sequence of tile partitions Few synchronization points (around 6) Serial and Parallel I/O (via NetCDF) Efficiency 4-64 threads

Horizontal mixing of tracers along level, geopotential, isopycnic surfaces Transverse, isotropic stress tensor for momentum General Length-Scale turbulence closure (GOTM) Local, Mellor-Yamada, level 2.5, closure scheme Non-local, K-profile, surface and bottom closure scheme SUBGRID-SCALE PARAMETERIZATION

Air-Sea interaction boundary layer from COARE (Fairall et al., 1996) Oceanic surface boundary layer (KPP; Large et al., 1994) Oceanic bottom boundary layer (inverted KPP; Durski, 2001) BOUNDARY LAYERS

Air-Sea interaction boundary layer from COARE (Fairall et al., 1996) Oceanic surface boundary layer (KPP; Large et al., 1994) Oceanic bottom boundary layer (inverted KPP; Durski et al., 2001) BOUNDARY LAYERS Wave / Current / Sediment bed boundary layer (Styles and Glenn, 2000)

Vertical Mixing and Sediment Models

Turbulence Sub-Models (parameterization of = -n t du/dz) Zero equation models – Prescribe n t One-equation models – n t ~ kl –Equation for k (advection/diffusion/production/dissipation) –Prescription for l Two-equation models –Equation for k –Equation for l Higher-order closures

MY25K-EK-W p m n Umlauf and Burchard (2002) Generic Length Scale Turbulence Model Eq. 1: Eq. 2:

Suspended sediment transport model Transport Erosion Deposition - consolidation - bioturbation Internal bed dynamics Water Sediment Bed

Constant bed slope Freshwater flow at east boundary Tidal elevation and vertical salinity gradient at west boundary Four turbulence models Estuary Test

Estuary Test: Salinity Front Warner and Sherwood (USGS) MY 2.5 GLS - KKL GLS - KE GLS - KW

Estuary Test: Suspended Sediment Warner and Sherwood (USGS) MY 2.5 GLS - KKL GLS - KE GLS - KW

Lagrangian Drifters (Klinck, Hadfield) Tidal Forcing (Hetland, Signell) River Runoff (Hetland, Signell, Geyer) Sediment erosion, transport and deposition (Warner, Sherwood, Blaas) Sea-Ice (Budgell, Hedstrom) Biology Fasham-type Model (Moisan, Di Lorenzo, Shchepetkin) EcoSim Bio-Optical Model (Bissett) MODULES

Models predict that surface particles travel offshore and bottom particles travel onshore during upwelling events Surface and sub-surface drifters (Rutgers-LEO) Red: Surface Blue: Bottom metersmeters

(PMEL/NOAA) Ocean currents transport fish larvae along the continental shelf in the Gulf of Alaska The Coastal Gulf of Alaska

Gulf of Maine M2 Tides Surface Elevation (m) 20 km Resolution (Hetland,Signell)

Tidal currents around Martha's Vineyard and Nantucket (Rutgers) Temperature (color) and currents (arrows)

Distance (km) Depth (m) Salinity (PSS) Hudson River Estuary Resolution 10 m (Hetland)

Southern California Bight ( ) SIO/UCSD

ROMS/TOMS: Tangent Linear and Adjoint Model Transformations

Tangent Linear and Adjoint Models There are many uses for tangent linear and adjoint models. The adjoint operator is a general sensitivity operator since it yields gradient information. In fact, any minimization or maximization that involves the linearized dynamical operators of a system will usually yield the adjoint equation. Thus, adjoint models are very useful for finding extrema.

Some uses of Adjoint Models 1.Data Assimilation Provides cost function gradient information. Can be used to fit model solutions to data by adjusting initial conditions, boundary conditions and parameters. 2.Sensitivity Analysis Sensitivity of a model solution to variations in model parameters can be evaluated very efficiently using adjoint models. 3.Eigenmode Analysis The eigenmodes of the tangent linear equations represent dynamic modes of variability (i.e. “normal modes”). These are our primary motivations for developing the tangent linear and adjoint versions of ROMS:

Some uses of Adjoint Models … 4.Adjoint Eigenmode Analysis: The eigenmodes of the adjoint equations represent the optimal excitations of the corresponding eigenmodes with respect to the chosen form. Most dynamical systems of interest are non-normal => Eigenmodes and adjoint eigenmodes are distinct. 5.Singular Vectors: These are the most rapidly growing perturbations that exist (linear limit) for the dynamical system with respect to chosen norm. Very useful for assessing the stability of the system. Very useful for perturbing the model initial conditions for generating ensembles of forecasts.

Some uses of Adjoint Models … 6.Stochastic Optimals (SOs) and Forcing Singular Vectors (FSVs): The most disruptive patterns of forcing for the TL model (with respect to a chosen norm). SOs assume forcing with zero time mean and stochastic in time. FSVs assume forcing that is time invariant. SOs and FSVs are useful for generating ensembles of forecasts. 7.Pseudospectra: Non-normal systems can display enhanced sensitivity and a large response to forcing frequencies that are far removed from resonant eigenmode frequencies. Pseudospectra are computed as the maximum singular values of the TL system resolvent.

The tangent linear model used in above is generally a different tangent linear model to that used in the IOM representer method. Two Tangent Linear Models, Same Adjoint

Consider the 1D advection equation: This is the tangent linear model for perturbations δ u that is generally used in 1-7. (1) Let Therefore,

(2) Alternatively, This is the tangent linear model used in the IOM representer Formulation.

FINAL REMARKS

Open Source Code Pro Non-exclusive Complete information transfer More users: more tests, wider range of applications, more tools,… More developers: more bug fixes, more processes, better algorithms,… Con More difficult to manage Requires volunteers Risk of misuse Intellectual property issues

Source Codes Pre- and Post-procesing software Forum, Chat Room, News, and Documentation SERVICES

Publications Ezer, T., H.G. Arango and A.F. Shchepetkin, 2002: Developments in Terrain-Following Ocean Models: Intercomparisons of Numerical Aspects, Ocean Modelling, 4, Haidvogel, D.B., H.G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A.F. Shchepetkin, 2000: Model Evaluation Experiments in the North Atlantic: Simulations in Nonlinear Terrain-Following Coordinates, Dyn. Atmos. Oceans, 32, MacCready, P. and W.R. Geyer, 2001: Estuarine Salt Flux through an Isoline Surface, J. Geoph. Res., 106, Malanotte-Rizzoli, P., K. Hedstrom, H.G. Arango, and D.B. Haidvogel, 2000: Water Mass Pathways Between the Subtropical and Tropical Ocean in a Climatological Simulation of the North Atlantic Ocean Circulation, Dyn. Atmos. Oceans, 32, Marchesiello, P., J.C. McWilliams and A.F. Shchepetkin, 2003: Equilibriom Structure and Dynamics of the California Current System, J. Phys. Oceanogr., 34, Marchesiello, P., J.C. McWilliams, and A.F. Shchepetkin, 2001: Open Boundary Conditions for Long-Term Integration of Regional Ocean Models, Ocean Modelling, 3, Moore, A.M., H.G. Arango, A.J. Miller, B.D. Cornuelle, E. Di Lorenzo, and D.J. Neilson, 2003: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, Ocean Modelling, Submitted. Peven, P., C. Roy, A. Colin de Verdiere and J. Largier, 2000: Simulation and Quantification of a Coastal Jet Retention Process Using a Barotropic Model, Oceanol. Acta, 23, Peven, P., J.R.E. Lutjeharms, P. Marchesiello, C. Roy and S.J. Weeks, 2001: Generation of Cyclonic Eddies by the Agulhas Current in the Lee of the Agulhas Bank, Geophys. Res. Let., 27, Shchepetkin, A.F. and J.C. McWilliams, 2003: The Regional Ocean Modeling System: A Split-Explicit, Free-Surface Topography-Following Coordinates Ocean Model, J. Comp. Phys., Submitted. Shchepetkin, A.F. and J.C. McWilliams, 2003: A Method for Computing Horizontal Pressure-Gradient Force in an Oceanic Model with a Non-Aligned Vertical Coordinate, J. Geophys. Res., 108, She, J. and J.M. Klinck, 2000: Flow Near Submarine Canyons Driven by Constant Winds, J. Geophys. Res., 105, Warner, J.C., H.G. Arango, C. Sherwood, B. Butman, and Richard P. Signell, 2003: Implementation and Applications of a Generic Length Scale Turbulence Closure and Suspended Sediment Transport Algorithms into the 3D Oceanographic Model ROMS, Ocean Modelling, Submitted.