CCA Data Object Contributors Ben Allan, Rob Armstrong, David Bernholdt, Robert Clay, Lori Freitag, Jim Kohl, Ray Loy, Manish Parashar, Craig Rasmussen, Andrew Siegel, Jaideep Ray, Lee Taylor

Working Group Meetings n SC00 (11/5/00) n First set of discussions n Created preliminary definitions n Identified issues relevant CCA n Set a schedule of meetings n Access Grid Meeting (11/30/00) n Review of SC00 meeting n ANL prototype for an unstructured discontinuous Galerkin application (L. Freitag) n Chicago FLASH center mesh database for block structured AMR calculations (A. Siegel) n Pre-CCA Meeting (12/6/00) (Afternoon) n TeraScale interfaces for unstructured finite element calculations (R. Clay) n Interfaces for raw data, arrays, fields (C. Rasmussen) n Grace’s interfaces for block structured AMR calculations (J. Ray)

Working Group Meetings (cont) n Pre-CCA Meeting (12/6/00) (Evening into the night) n Group discussion of “The Big Picture” n Data Managers, Raw Data, Arrays, Fields, Meshes n Subgroup discussions until the wee hours n CCA Meeting (12/7/00) n Presentation of the dicussions to date n Looking for feedback on preliminary design ideas n Overview: L. Freitag n Raw Data and Arrays: B. Allan n Unstructured Meshes and Fields: R. Clay n Disclaimer n These are initial thoughts that are only partially complete in some cases n Everything presented is open for discussion

SC00 Definitions n Field: a scalar, vector, or strided combination of scalars and vectors defined with respect to a geometric extent n e.g. Pressure defined on mesh vertices n FieldSet: a collection of fields n e.g. a hydro field set containing pressure, velocity and energy fields n FieldSets can contain other instances of FieldSets n e.g. an application FieldSet containing a hydro field set and a chemical species FieldSet n Mesh: a discrete representation of a spatial domain n collection of uniquely identifiable vertices, edges, faces, and cells n relationship among entities given by hierarchical and connectivity information n Meshes can contain other instances of Meshes

Issues Relevant to CCA (as discussed at SC00) Structured and Unstructured Meshes n There are differences in how structured and unstructured meshes store and use their data n Structured grids commonly use n-dimensional arrays n Unstructured grids may use n-dimensional arrays but also commonly use linked lists n Allocation and access patterns can be quite different so different interfaces are required for performance n Interfaces defined by CCA should not dictate how the data be stored, particularly data does not need to be allocated as a single memory block n Only minimal sets of interfaces should be required - extensions can be supported for performance, etc.

Parallel Implementation Issues n Decompositions must be able to be determined dynamically n most thought this functionality should be decoupled from the mesh/field data object n A partitioned mesh must have two capabilities n it should be able to serialize itself n it should be able to update ghost nodes n Interfaces for parallel meshes must support n update of ghost cells n parallel I/O to files n perhaps done using a string-based option in the interface n underlying implementations such as collect first then write or write then serialize should be identifiable in the API

Interfaces n Level 0: (Required) n Data Representations n Currently in the form of Raw Data, Arrays, Fields and Meshes, n Each must accommodate different data types n Data I/O n Should be able to define and access subsets of data n Should be able to define, access, and use subsets of meshes n Data Management n Level 1: (Desired but not Required) n “Update” of ghost nodes n Should this be level 0? (Rob thinks so) n Should this be at the decomposition level or at the data level n Collective operations over fields such as max, min, sup, etc n Mesh modification APIs (adaptation, smoothing, flipping) n Redistribution (is this the mxn problem?)

Level 0 Interfaces n Raw Data Port n Set of smart pointers containing name, location, pointer, size, type, strides describing a strip of memory n collection of 1D buffers each of which can have a different type n Modeled on unix IOvecs n Low level and general n can therefore be used as a handle for moving data efficiently n Considered to be local only n Array Port n chunk of memory arranged topologically (1d,2d,3d) n contains name, dimension, shape n should be able to return subsets of raw data n can be SPMD

Level 0 Interfaces n Fields and Meshes n Fields are defined on mesh extents n Fields have a math type (scalar, vector, quaternions,…) n Fields can be implemented as Arrays n Meshes can be structured or unstructured n Both require nodes with coordinate information and dimension information n Unstructured additionally required elements with connectivity information n Fields and meshes are inherently related (one is no good without the other) n SPMD

Level 0 Interfaces n DataManager n Container for named meshes, fields, arrays and raw data n names must be unique n Job is to marshal data (e.g. to disk or to components needing data) n Data access is via string name n Components that operate on data interact with the DataManager to obtain the data n Multiple managers can be wired to a component n Implies a registry to prevent name conflict DM1 DM2 Raw Data Array Structured Mesh Unstructured Mesh CFD FFT MxN VIZ T

Design Procedure To list anticipated operations in the data interface design and to test it against existing applications and mesh field data implementations DISCUSSION?

OLD STUFF

Relationships n CCAArrays contain raw data n extension of raw data in that it contains dmenions and is decomposable n Data Managers can contain fields and meshes, cca arrays and raw data n raw data must be associated with a named entity to be served up by a data manager n Getting subregions of raw data rquired knowing what object the raw data lives on (how else can you define the subregion)

Interface Ideas from ANL: Background n Existing mesh management package: SUMAA3d n Targeted at Finite Element Applications n Handles n Mesh Infrastructure Management n Mesh Modifications (adaptation, coarsening, smoothing, flipping) n Uses URB partitioning (tightly coupled to mesh data structures) n Only simple geometry and boundary definition n Primary data structures are linked lists of elements, edges, or vertices n User Data can be user-defined via subroutine callbacks; accessed on entity by entity basis n Attempting to define interfaces that decouple a discontinuous Galerkin application from SUMAA3d data structures

Design Philosophy n Interfaces are defined in small groups of related functionality n Atomic operations required n Additional interfaces for performance or specialized operations n The required interface set must be small to encourage adoption by mesh management tool providers n Not all interfaces need be implemented by all mesh management packages n Defined interfaces for Vertices, Edges, and Elements n Some geometric and topological information n Simple data I/O that assumed type double n By no means complete - prototype only for testing performance n Original goal is to minimize changes to SUMAA3d

General Mesh Methods Class MeshComponent { MeshComponent(); ~MeshComponent(); int setOption(char *, int); // also for float, double, char int getOption(char *, int *); // also for float, double, char int publishOptions(); int setNumberofVertexUnknowns(int); // also Edge, Element int createMesh(); int intializeUserData(); int getNumberOfVertices(int *); // also Edge, Element int getMeshVertexList(VertexList *); // also Edge, Element }

Vertex Methods Class Vertex { Vertex(); ~Vertex(); int getID(int *); int getVertexCoordinates(double **); int getUserData(VertexUserData *); int setUserData(VertexUserData); int getNextVertex(); }

Edge Methods Class Edge { Edge(); ~Edge(); int getID(int *); int getVertices(Vertex **); int getElements(Element **); int getUserData(EdgeUserData *); int setUserData(EdgeUserData); int getNextEdge(Edge *); }

Element Methods Classs Element { Element(); ~Element(); int getID(int *); int getVertices(Vertex **); int getUserData(ElementData *); int setUserData(ElementData); int getNextElement(Element *); }

ElementData Methods Class ElementData { ElementData(); ~ElementData(); int getNumData(int *); int setNumData(int); int getData(double **); int setData(double *); int getData(int num, int *index, double **data); int setData(int num, int *index, double *data); } EdgeData and VertexData are similiar

ElementList Methods Class ElementList { ElementList(); ~ElementList(); int append(Element); int prepend(Element); int delete(Element); } VertexList and EdgeList are similar