Cactus Workshop - NCSA Sep 27 - Oct Adding a Driver Thorn to Cactus Tom Goodale Albert Einstein Institute

n Allocates all memory of all grid variables l Flesh doesn't know how things are stored in memory n Communicates grid variables l Flesh doesn't know what parallisation, if any, is in operation n May take charge of the evolution l Flesh knows nothing about multiple patches, AMR, etc. What does a Driver do?

Anatomy of a driver n Startup n GH Extension n Memory allocation/deallocation n Communication enabling/disabling n Evolution n Reduction n Interpolation

Startup n Must register itself as a GH Extension l Adds data to the GH structure n Overload functions l Synchronisation and communication l Memory allocation l Parallel functions l Evolution Step n May register reduction operators n May register interpolation functions

GH Extensions n Used to associate data with a GH n Three functions l SetupGH – Returns the memory structure to hold data l InitGH – Initialises the data structure l ScheduleTraverse – Used for traversing the scheduling trees

Memory n The flesh provides information about the types of variables, but doesn't, itself, store allocate any memory for them. n A driver must overload l CCTK_EnableGroupStorage l CCTK_DisableGroupStorage n Driver can store the grids however it likes, as long as thorns are passed arrays in Fortran order of size cctk_sh(1),... n Also must overload query functions which return the status of the storage, and the size in each direction

Communication n The Driver must provide functions to synchronise the variable groups, so must overload l CCTK_SyncGroup l CCTK_EnableGroupComm l CCTK_DisableGroupComm n The flesh provides default empty ones for these if there is no parallelism.

n The flesh provides a simple unigrid evolution stepper. This assumes one grid. n A driver could support multiple grids, in which case it needs to overload the CCTK_Evolution routine. n On each grid it should make sure it calls CCTK_Schedule(GH, point) on l CCTK_PRESTEP l CCTK_EVOL l CCTK_BOUND l CCTK_POSTSTEP l CCTK_ANALYSIS n The timestep should be updated after bound. Evolution

Reduction and Interpolation n Some thorns need to do global operations like getting the maximum of a grid function across all grids. It provides these by registering reduction operators, and should at least define the maximum, minimum, L1 and L2 norms. n Additionally, since a driver is in principle the only thing which needs to know the grid structure, it should provide a means of getting the value of a grid function at a particular point in coordinate space irrespective of which processor it's on. This can be provided by registering an interpolation routine.