1. Hernan G. Arango, Rutgers University (arango@imcs.rutgers.edu) Tal Ezer, Pricenton University (ezer@splash.princeton.edu) FTP File: TOMS.tar A Community T errain-following O cean Modeling S ystem

2. YTo design, develop and test an expert ocean modeling system for scientific and operational applications YTo support advanced data assimilation strategies YTo provide a platform for coupling with operational atmospheric models (like COAMPS) YTo support massive parallel computations YTo provide a common set of options for all coastal developers with a goal of defining an optimum coastal/relocatable model for the navy OBJECTIVES

3. YUse state-of-the-art advances in numerical techniques, subgrid-scale parameterizations, data assimilation, nesting, computational performance and parallelization YModular design with ROMS as a prototype YTest and evaluate the computational kernel and various algorithms and parameterizations YBuild a suite of test cases and application databases YProvide a web-based support to the user community and a linkage to primary developers APPROACH

4. “The complexity of physics, numerics, data assimilation, and hardware technology should be transparent to the expert and non-expert USER” CHALLENGE

5. YFree-surface, hydrostatic, primitive equation model YGeneralized, terrain-following vertical coordinates YBoundary-fitted, orthogonal curvilinear, horizontal coordinates on an Arakawa C-grid YNon-homogeneous time-stepping algorithm YAccurate discretization of the baroclinic pressure gradient term YHigh-order advection schemes YContinuous, monotonic reconstruction of vertical gradients to maintain high-order accuracy TOMS KERNEL ATTRIBUTES

6. Dispersive Properties of Advection  /2  /4 3  /4 kxkx 1/2 1 3/2 2 5/2 K(k)  x 2 4 6 10 8 Parabolic Splines Vs Finite Centered Differences

7. YHorizontal mixing of tracers along level, geopotential, isopycnic surfaces YTransverse, isotropic stress tensor for momentum YLocal, Mellor-Yamada, level 2.5, closure scheme YNon-local, K-profile, surface and bottom closure scheme TOMS SUBGRID-SCALE PARAMETERIZATION

8. YAir-Sea interaction boundary layer from COARE (Fairall et al., 1996) YOceanic surface boundary layer (KPP; Large et al., 1994) YOceanic bottom boundary layer (inverted KPP; Durski et al., 2001) TOMS BOUNDARY LAYERS

10. YAir-Sea interaction boundary layer from COARE (Fairall et al., 1996) YOceanic surface boundary layer (KPP; Large et al., 1994) YOceanic bottom boundary layer (inverted KPP; Durski et al., 2001) TOMS BOUNDARY LAYERS YWave / Current / Sediment bed boundary layer (Styles and Glenn, 2000) YSediment transport

11. YLagrangian Drifters (Klinck, Hadfield) YTidal Forcing (Hetland, Signell) TOMS MODULES

12. Gulf of Maine M2 Tides Surface Elevation (m)

13. YLagrangian Drifters (Klinck, Hadfield) YTidal Forcing (Hetland, Signell) TOMS MODULES YRiver Runoff (Hetland, Signell, Geyer)

14. 5 10 1520 25 Distance (km) -10 -15 -25 -20 -5 Depth (m) 30 25 20 15 10 5 Salinity (PSS) Hudson River Estuary

15. YLagrangian Drifters (Klinck, Hadfield) YTidal Forcing (Hetland, Signell) Y River Runoff (Hetland, Signell, Geyer) TOMS MODULES YBiology Fasham-type Model (Moisan, Shchepetkin) YEcoSim Bio-Optical Model (Bissett)

16. YModular, efficient, and portable Fortran code (F77+, F90) YC-preprocessing managing YMultiple levels of nesting YLateral boundary conditions options for closed, periodic, and radiation YArbitrary number of tracers (active and passive) YInput and output NetCDF data structure YSupport for parallel execution on both shared- and distributed -memory architectures TOMS CODE DESIGN

17. YCoarse-grained parallelization TOMS PARALLEL DESIGN

18. } } Nx Ny PARALLEL TILE PARTITIONS 8 x 8

19. YCoarse-grained parallelization TOMS PARALLEL DESIGN YShared-memory, compiler depend directives MAIN (OpenMP standard) YDistributed-memory (MPI; SMS) YOptimized for cache-bound computers YZIG-ZAG cycling sequence of tile partitions YFew synchronization points (around 6) YSerial and Parallel I/O (via NetCDF) YEfficiency 4-64 threads

20. YNudging YOptimal Interpolation (OI) YTangent linear and Adjoint algorithms Y4D VARiational data assimilation (4DVAR) and Physical Statistical Analysis System (PSAS) algorithms YInverse Ocean Modeling System (IOMS) YEnsemble prediction platform based on singular value decomposition YError Subspace Statistical Estimation (ESSE) TOMS DATA ASSIMILATION

21. Historical, Synoptic, Future in Situ/Remote Field/Error Observations d 0 R 0 Field Initialization Central Forecast Sample Probability Density Mean Select Best Forecast Shooting ESSE Smoothing via Statistical Approximation Minimum Error Variance Within Error Subspace (Sequential processing of Observations) Measurement Model A Posteriori Residules dr (+) Performance/ Analysis Modules OA via ESSE Gridded Residules Synoptic Obs Measurement Model Measurement Error Covariance   ^  cf (-) ^     00 Options/ Assumptions Most Probable Forecast  mp (-) ^ Ensemble Mean  q {  j  ^ Adaptive Error Subspace Learning Convergence Criterion Continue/Stop Iteration Breeding Peripherals Analysis Modules Normalization SVD p Continuous Time Model Errors Q(t) Scalable Parallel Ensemble Forecast + PerturbationsError Subspace Initialization 1jq1jq    1jq1jq ^ ^ ^ u j (o,Ip) with physical constraints +  (+) ^ E(+)  (+) E0E0  (+) ^ Ea(+)a(+)Ea(+)a(+) Field Operation Assumption Key  (-) ^ E(-)  (-) - + + + - + - - - - - + + + d  C  (-) Data Residuals ^ + + + - - +/- + j=1 j=q + ESSE Flow Diagram

22. YDensity Jacobian Class (Blumberg and Mellor, 1987; Song 1998; Song and Wright 1998) u More Accurate u Error vanishes with linear density profiles YPressure Jacobian Class (Lin 1998; Shchepetkin and McWilliams, 2001) u JEBAR consistent u Conserve Energy PRESSURE GRADIENT FORCE

23. Seamount Test Case (64 x 64 x 20) dx = dy = 8 km

24. Models with 2nd order advection scheme POM ROMS Second Order Advection Scheme Surface Elevation Anomaly Stream Function Anomaly

25. Advection Schemes in ROMS (Seamount Case) V Second Order Centered Third Order Upstream Bias Fourth Order Centered

26. Pressure Gradient Errors POM (6th order) U (cm/s) V (cm/s) ROMS POM X (km)

27. Relative CPU per time step Percentage

28. XBuild TOMS from ROMS prototype uMellor-Yamada, level 2.5 uPassive and active open boundary conditions uTidal forcing uRiver runoff uLagrangian drifters uData assimilation XInter-comparison between POM and ROMS uEvaluation of time-stepping, advection, and pressure gradient algorithms XInitial development of TOMS web site RESULTS (YEAR 1)

29. YBennett et al. (FNMOC; OSU) YChassignet / Iskandarani et al. (RSMAS) YCornuelle / Miller (SIO) YGeyer (WHOI) YHetland (TAMU) YLermusiaux (Harvard) YMellor (Pricenton) YMoore (U. Colorado) YShchepetkin (UCLA) YSignell (SACLANT; USGS) COLLABORATORS

30. YChao / Song (JPL) YPreller / Martin (NRL) YNaval Operational Community YPOM Ocean Modeling Community YROMS / SCRUM Ocean Modeling Community OTHER COLLABORATORS

31. YTo Be Determined !!! YPotential Users: uNAVO uFNMOC uNOAA uUSCG TRANSITION PATHS

32. XChassignet et al., 2000: Damee modeling review XEzer, 2000: Mixed-layer evaluation XEzer and Mellor, 2000: POM Damee application XHaidvogel et al., 2000: ROMS Damee application XMalanotte-Rizzoli et al., 2000: ROMS Damee XMellor, 2001: Improved turbulence scheme XMellor et al., 2001: Generalized vertical coordinate PUBLICATIONS

33. Initial web page: www.aos.princeton.edu/WWWPUBLIC/ezer/TOMS