1. kajny@ida.liu.se GridModelica High Level Modeling on the Grid Kaj Nyström Dept. of Computer and Information Science, Linköping University MathCore Engineering AB kajny@ida.liu.se

2. Modeling – The way of doing things Modeling (like talk) is cheap, considering the alternatives Modeling is almost always possible Modeling gives extensive data even where physical measuring might not always be possible, or is too expensive. Modeling is fast, especially on large systems Limiting factors: Model complexity Computational power

3. kajny@ida.liu.se Modeling – The way of doing things(2) Factors limiting usage of modeling Conservative engineers Domain knowledge Model complexity Computational power

4. kajny@ida.liu.se Modeling – Dealing with complexity Traditionally high performance models have been written in C or Fortran. The Modelica language provides an object oriented approach to modeling. Main benefits from our point of view: Acausal: No need to do the mathematical dirtywork yourself anymore. Connection oriented: Well defined interface between all components (very useful for parallelization). Reusable: Extensive object libraries exists.

5. kajny@ida.liu.se Modeling – Dealing with complexity(2) Example: The DC-motor

6. kajny@ida.liu.se Modeling – Dealing with complexity(3) Example: The DC-motor, modelica source code model dcmotor Modelica.Electrical.Analog.Basic.Resistor Resistor1; Modelica.Electrical.Analog.Basic.Ground Ground1; Modelica.Electrical.Analog.Basic.Inductor Inductor1; Modelica.Electrical.Analog.Basic.EMF EMF1; Modelica.Electrical.Analog.Sources.ConstantVoltage ConstantVoltage1; Modelica.Electrical.Analog.Basic.Resistor Resistor2(R=10); equation connect(Resistor1.p, ConstantVoltage1.p) ; connect(Resistor1.n, Inductor1.p); connect(Inductor1.n, EMF1.p); connect(ConstantVoltage1.n, Ground1.p); connect(EMF1.n, Ground1.p); connect(EMF1.flange_b, Inertia1.flange_a); connect(Resistor2.p, Resistor1.p); connect(Resistor2.n, Resistor1.n); end dcmotor;

7. kajny@ida.liu.se Modeling – Dealing with complexity(4) Another example: A robot, including visualization

8. kajny@ida.liu.se Problem 2: Computational Power The Grid is the solution (?) – Computational power for the masses. Nordugrid middleware Resources: Nordugrid and our own cluster This provides the computational power we need but it also just brings us to the next (big) problem:

9. kajny@ida.liu.se Problem no 2: Computational Power The Grid is the solution (?) – Computational power for the masses. Nordugrid middleware Resources: Nordugrid and our own cluster This provides the computational power we need but it just brings us to the next (big) problem: Partitioning!

10. kajny@ida.liu.se An example: a+b+c+d=10 a+b+c=2 c-d=-2 a+b-d=4 How would you solve this system? Partitioning models on the Grid

11. kajny@ida.liu.se The grid has special features/misfeatures: Latency, latency, latency, latency… Heterogenity Close to infinite resources Unpredictability Unreliability Partitioning models on the Grid (2)

12. kajny@ida.liu.se We attack the problem on different levels High level partitioning - New Modelica constructs to enable parallelism. Medium level partitioning - Shared level memory model (GridNestStep). Low level partitioning – Parallelizing the generated C-code. Partitining the models

13. kajny@ida.liu.se Partitioning at Modelica model level Done by adding partitioning constructs to the language and modification of the Modelica compiler. Considered methods are: Weak Operators/Variables (Francesco Casella) Transmission Line Modeling method (Christos Christopoulos) Modelica specific ideas High Level Partitioning

14. kajny@ida.liu.se High Level Partitioning (2) The weak variables method 1. Partition the model into physical domains 2. Identify boundary variables 3. Treat these variables as parameters on both sides of boundary. 4. Use fast fixed step solvers but with different step- size for both subsystems

15. kajny@ida.liu.se The transmission line modeling method 1. Partition the model fast and slow subsystems 2. Insert delaying TLM-element between subsystems 3. Calculate approximate loss in precision and compensate if possible High Level Partitioning (3)

16. kajny@ida.liu.se Grid level parallelization with GridNestStep Shared memory model for the grid Specific adaption for the grid is neccesary, maximize time between syncs is absolutely vital. Medium Level Partitioning

17. kajny@ida.liu.se Primarily by task merging and common parallelization techniques in parallel solvers Task merging: Partition code very fine grained Achive parallelism by a sequence of duplicating tasks and merging very small jobs, depending on computational and communication cost. This can be done adaptively for large jobs Low Level Partitioning

18. kajny@ida.liu.se Parallelizing solving of large equation systems is hard. Latency is our worst enemy Different variants of speculative execution might be our best friends The grid might be a hype, but it is a useful hype. Wrap up

19. kajny@ida.liu.se Thank you for your attention Questions?