1. Technische Universität München Peano 2

2. Technische Universität München Peano 2, Jun 13 2011 - 2 - Features of Peano Adaptive Cartesian Grids –Arbitrary adaptivity –Arbitrary dimensions (2D/3D/...) –Dynamic refinement & coarsening –Multiscale representation Memory Behaviour –Very low memory requirements –Good cache behaviour Parallelisation –Shared and/or distributed –memory parallelisation –Domain decomposition model –In this presentation, we dicuss how to exploit these features.

3. Technische Universität München Peano 2, Jun 13 2011 - 3 - Agenda Peano's Principles – k-spacetrees – Grid traversal & call-back mechanism Peano Architecture – Philosophy and terminology – T-Architecture – Unit tests – Configuration files – Grid setup PeProt: Automatic Code Generation – Model the degrees of freedom – Introducing behaviour

4. Technische Universität München Peano 2, Jun 13 2011 - 4 - k-Spacetrees Generalisation of Octrees Construction –Embed domain into unit square (domain) –Refine unit square: Domain is contained in central element –Continue recursively –Store only tree Remarks –Any dimension supported –k=3 for Peano space-filling curve –Low memory requirements –Multiple vertices at same position –Hanging nodes

5. Technische Universität München Peano 2, Jun 13 2011 - 5 - k-Spacetrees Generalisation of Octrees Construction –Embed domain into unit square (domain) –Refine unit square: Domain is contained in central element –Continue recursively –Store only tree Remarks –Any dimension supported –k=3 for Peano space-filling curve –Low memory requirements –Multiple vertices at same position –Hanging nodes

6. Technische Universität München Peano 2, Jun 13 2011 - 6 - k-Spacetrees Generalisation of Octrees Construction –Embed domain into unit square (domain) –Refine unit square: Domain is contained in central element –Continue recursively –Store only tree Remarks –Any dimension supported –k=3 for Peano space-filling curve –Low memory requirements –Multiple vertices at same position –Hanging nodes

7. Technische Universität München Peano 2, Jun 13 2011 - 7 - k-Spacetrees Generalisation of Octrees Construction –Embed domain into unit square (domain) –Refine unit square –Continue recursively –Store only tree Remarks –Any dimension supported –k=3 for Peano space-filling curve –Low memory requirements –Multiple vertices at same position –Hanging nodes

8. Technische Universität München Peano 2, Jun 13 2011 - 8 - Grid Traversal Traversal Paradigms –Each solver has to run over grid –Iterator concept (for loop) vs. –call-back mechanism Call-back with Traversal Events –How-to traverse is encapsulated –Where to go next is encapsulated –Each grid transition triggers event –User can plug into events Limitations –Never traverse subsets of grid –User can’t control traversal direction –or behaviour, but –complexity of traversal is hidden and –it might run in parallel. Transition triggers an event

9. Technische Universität München Peano 2, Jun 13 2011 - 9 - Grid Traversal Traversal Paradigms –Each solver has to run over grid –Iterator concept (for loop) vs. –call-back mechanism Call-back with Traversal Events –How-to traverse is encapsulated –Where to go next is encapsulated –Each grid transition triggers event –User can plug into events Limitations –Never traverse subsets of grid –User can’t control traversal direction –or behaviour, but –complexity of traversal is hidden and –it might run in parallel. Transition triggers an event, and this Event is mapped to (several) event handles – the user-defined functions void touchVertexFirstTime( Vertex &vertex) { […] vertex.refine(); […] }

10. Technische Universität München Peano 2, Jun 13 2011 - 10 - Agenda Peano's Principles – k-spacetrees – Grid traversal & call-back mechanism Peano Architecture – Philosophy and terminology – T-Architecture – Unit tests – Configuration files – Grid setup PeProt: Automatic Code Generation – Model the degrees of freedom – Introducing behaviour

11. Technische Universität München Peano 2, Jun 13 2011 - 11 - Peano Architecture

12. Technische Universität München Peano 2, Jun 13 2011 - 12 - Peano – Philosophy and Terminology Philosophy –Algorithm = sequence of phases (implemented in runner) –Phase consists of every computing node receives global state, grid is traversed once (in parallel), and global state is reduced. Repository –Repository = big box hiding Peano kernel and phase management –Single point of contact for algorithm (SPoC) –Two types of operations: iterate() and switchToPhaseXXX()

13. Technische Universität München Peano 2, Jun 13 2011 - 13 - What happens when I call iterate()? Terminology & Concept –Repository tells grid to traverse with automaton –Transition of automaton triggers event –Users can plug into events, i.e. behaviour is mapping from events to operations –Algorithm selects phase, i.e. a set of mappings to apply to traversal –Such a combination is an adapter What defines a Peano Application? –Data stored on the grid –Global state –Mappings –Adapters = combinations/merge of mappings

14. Technische Universität München Peano 2, Jun 13 2011 - 14 - T-Architecture: The namespace tarch Peano’s Principles –Global constants, assertions –Log statements –xml parsing utilities –Linear algebra routines –Test environment –Output routines (vtk/Paraview, TecPlot) Rationale –Direct usage (just include it) –Static linked –Own namespace (not peano) as other apps use it, too (preCICE, e.g.) #include ‘‘tarch/logging/Log.h‘‘ class peano::applications::poisson::AbstractSolver { private: static tarch::logging::Log _log; protected: [... ] void peano::applications::poisson::AbstractSolver::foo() { [... ] _log.debug( ‘‘ foo ‘‘(), ‘‘ this is just a log ‘‘ ); [... ]

15. Technische Universität München Peano 2, Jun 13 2011 - 15 - Automatic testing –Testing framework is part of T-architecture –Each application defines set of tests –Tests are executed by all developers all the time Implementation –Create subclass of tarch::TestCase –Add run method and register test –Make run method (or submethods) and use validate macros –Run code with test configuration file Unit Tests void MyTestCase::run() { void testGetters(); } void MyTestCase::testGetters() { [...] validateEquals( myObject.getSomething(), 1.2, ‘‘testGetters()‘‘ ); [...] } validateEquals in line xxx in file MyTestCase.cpp failed: 0.2 instead of 1.2. Running test case collection peano.kernel.spacetreegrid.tests......... ok Running test case collection MyTestCase......... failed Runnint test case collection unit tests......... failed

16. Technische Universität München Peano 2, Jun 13 2011 - 16 - One Config File Policy –One xml file configures application (no other arguments) –Each top level tag defines one program run –Unit Tests equal one program run Structure –Logging tags –Test tag –(Main) Application tag App-specific (sub-)tags Spacetree-/ Regulargrid subtag Geometry subtag Plotter/ output subtags –etc. Configuration Files <log-output column-separator=" " log-time-stamp="yes" log-time-stamp-human-readable="yes" log-machine-name="no" log-message-type="yes" log-trace="yes" />... <spacetree-grid number-of-vertices-on-reference-level="18 ; 18 ; 18" number-of-simulation-grid-levels="2" domain-size="1.0 ; 1.0 ; 1.0" domain-offset="0.0 ; 0.0 ; 0.0" />...

17. Technische Universität München Peano 2, Jun 13 2011 - 17 - Technical principles –No difference between grid construction and other traversals –Grid creation events –Every user steers grid construction by –him-/ herself Math –Computational domain shrinks to computational grid –Each refinement at boundary makes grid grow to continuous domain –Three types of vertices: inside, outside, boundary. Special callbacks –for each vertex creation. –Two types of cells: inside, outside –Use peano::grid::GeometryHelper Grid Setup

18. Technische Universität München Peano 2, Jun 13 2011 - 18 - Technical principles –No difference between grid construction and other traversals –Grid creation events –Every user steers grid construction by –him-/ herself Math –Computational domain shrinks to computational grid –Each refinement at boundary makes grid grow to continuous domain –Three types of vertices: inside, outside, boundary. Special callbacks –for each vertex creation. –Two types of cells: inside, outside –Use peano::grid::GeometryHelper Grid Setup void MyMapping::createBoundaryVertex(MyVertex& arg) { [...] // initialise values within this vertex: // set Dirichlet values etc. [...] }

19. Technische Universität München Peano 2, Jun 13 2011 - 19 - Agenda Peano's Principles – k-spacetrees – Grid traversal & call-back mechanism Peano Architecture – Philosophy and terminology – T-Architecture – Unit tests – Configuration files – Grid setup PeProt: Automatic Code Generation – Model the degrees of freedom – Introducing behaviour

20. Technische Universität München Peano 2, Jun 13 2011 - 20 - PeProt: Automatic Code Generation PeProt scripting –Define your data structures –Define application-specific mappings –Define mapping combinations as adapters –Stencil evaluations: Simple definition and automatic code generation for these –types of vertex-/ cell-events component: multigrid configuration-tag: poissonmultigrid repository: MultigridBatchJob namespace: peano::applications::poisson::multigrid vertex: name: Vertex dastgen-file: Vertex.def read scalar: Residual read scalar: U read stencil: Stencil read vector2PowD: TempP read vector2PowD: TempAP write scalar: Residual write stencil: Stencil write vector2PowD: TempP write vector2PowD: TempAP... event-mapping: name: SetupExperiment event-mapping: name: PlotSolution … adapter: name: SetupExperimentAndPlotStartSolution merge-with-user-defined-mapping: SetupExperiment merge-with-user-defined-mapping: PlotSolution

21. Technische Universität München Peano 2, Jun 13 2011 - 21 - Model the Degrees of Freedom Application Model –DaStGen – similar to C++ –Automatically adds getter, setter, MPI, etc. –Automatically reduces memory footprint –For both vertices and cells Grid-Vertex/-Cell Superclass –Let DaStGen application models inherit –from these superclasses –Hold refinement status, parallelisation information, etc. –Hold flags for hanging nodes Integration –DaStGen performs hierarchy flattening (reduce memory footprint further) –Put generated records in a records subdirectory –Create your (vertex-) operations in user-defined classes –Instantiate grid with your vertices/cells as type argument Vertex holds properties Cell holds properties

22. Technische Universität München Peano 2, Jun 13 2011 - 22 - Model the Degrees of Freedom Vertex holds properties Cell holds properties Application Model –DaStGen – similar to C++ –Automatically adds getter, setter, MPI, etc. –Automatically reduces memory footprint –For both vertices and cells Grid-Vertex/-Cell Superclass –Let DaStGen application models inherit –from these superclasses –Hold refinement status, parallelisation information, etc. –Hold flags for hanging nodes Integration –DaStGen performs hierarchy flattening (reduce memory footprint further) –Put generated records in a records subdirectory –Create your (vertex-) operations in user-defined classes –Instantiate grid with your vertices/cells as type argument #include "peano/utils/Globals.h" #include "tarch/logging/Log.h" Packed-Type: short int; class peano::applications::poisson::records::PoissonCellDoF { enum Type { UNDEF, INNER, BOUNDARY }; persistent packed Type _type; };

23. Technische Universität München Peano 2, Jun 13 2011 - 23 - Introducing Behaviour Starting Point –Cells, vertices, mappings, runners etc. have been generated –Cell- and vertexwise operations are implemented –There’s a configuration class for new solver Mapping –Open generated mapping –Trigger operations within the mapping –callbacks –Simulation State: DaStGen-generated record, –holding the state information of the solver –→ relaxation parameters, number of timesteps... Integrate –Instantiate grid (from configuration, geometry...) –Trigger adapter calls within runner: One line of code! void MyMapping::enterCell( Cell& cell, Vertices* vertices ) { [...] if ( cell.isSolverCell() && vertices[0].isPersistent() ) { _solver.evaluateStencil( cell,vertices ); } [...] } void MyRunner::run(){... myAdapterRepository.switchToMySolverAdapter(); myAdapterRepository.iterate(); }

24. Technische Universität München Peano 2, Jun 13 2011 - 24 - Conclusion & Features revisited Peano 2: Grid enviroment for adaptive multiscale descriptions – PDE solvers (finite elements, finite differences) – Multilevel/ Multigrid schemes – Flexible geometry discretisation Adaptive Cartesian Grids –Arbitrary adaptivity –Arbitrary dimensions (2D/3D/...) –Dynamic refinement & coarsening Memory Behaviour –Very low memory requirements –Good cache behaviour Parallelisation –Shared and/or distributed –memory parallelisation –Domain decomposition model

25. Technische Universität München Peano 2, Jun 13 2011 - 25 - Current Projects Persons in charge Tobias Weinzierl Tobias Neckel Janos Benk Bernhard Gatzhammer Michael Lieb Philipp Neumann Marion Bendig Kristof Unterweger Atanas Atanasov CFD (Navier-Stokes) FSI Heat Equation Turbulent Flows Chemical Reactions Framework’s use depends on applications Applications’ feedback improves framework Lattice Boltzmann Poisson Eq. - Multigrid Sundance coupling Computational steering Shallow Water Equations

26. Technische Universität München Peano 2, Jun 13 2011 - 26 - Thank you!