CCA Data Object

ContributorsBen Allan, Rob Armstrong, David Bernholdt, Robert

Clay, Lori Freitag, Jim Kohl, Ray Loy, Manish Parashar,

Craig Rasmussen, Andrew Siegel, Jaideep Ray, Lee Taylor

Working Group Meetings SC00 (11/5/00)

First set of discussions Created preliminary definitions Identified issues relevant CCA Set a schedule of meetings

Access Grid Meeting (11/30/00) Review of SC00 meeting ANL prototype for an unstructured discontinuous Galerkin

application (L. Freitag) Chicago FLASH center mesh database for block structured AMR

calculations (A. Siegel) Pre-CCA Meeting (12/6/00) (Afternoon)

TeraScale interfaces for unstructured finite element calculations (R. Clay)

Interfaces for raw data, arrays, fields (C. Rasmussen) Grace’s interfaces for block structured AMR calculations (J. Ray)

Working Group Meetings (cont)

Pre-CCA Meeting (12/6/00) (Evening into the night) Group discussion of “The Big Picture”

Data Managers, Raw Data, Arrays, Fields, Meshes Subgroup discussions until the wee hours

CCA Meeting (12/7/00) Presentation of the dicussions to date Looking for feedback on preliminary design ideas Overview: L. Freitag Raw Data and Arrays: B. Allan Unstructured Meshes and Fields: R. Clay

Disclaimer These are initial thoughts that are only partially complete in

some cases Everything presented is open for discussion

SC00 Definitions

Field: a scalar, vector, or strided combination of scalars and vectors defined with respect to a geometric extent e.g. Pressure defined on mesh vertices

FieldSet: a collection of fields e.g. a hydro field set containing pressure, velocity and

energy fields FieldSets can contain other instances of FieldSets

e.g. an application FieldSet containing a hydro field set and a chemical species FieldSet

Mesh: a discrete representation of a spatial domain collection of uniquely identifiable vertices, edges, faces,

and cells relationship among entities given by hierarchical and

connectivity information Meshes can contain other instances of Meshes

Issues Relevant to CCA (as discussed at SC00)

Structured and Unstructured Meshes

There are differences in how structured and unstructured meshes store and use their data Structured grids commonly use n-dimensional arrays Unstructured grids may use n-dimensional arrays but also

commonly use linked lists Allocation and access patterns can be quite different so

different interfaces are required for performance Interfaces defined by CCA should not dictate how

the data be stored, particularly data does not need to be allocated as a single memory block

Only minimal sets of interfaces should be required - extensions can be supported for performance, etc.

Parallel Implementation Issues

Decompositions must be able to be determined dynamically most thought this functionality should be decoupled

from the mesh/field data object A partitioned mesh must have two capabilities

it should be able to serialize itself it should be able to update ghost nodes

Interfaces for parallel meshes must support update of ghost cells parallel I/O to files

perhaps done using a string-based option in the interface underlying implementations such as collect first then

write or write then serialize should be identifiable in the API

Interfaces

Level 0: (Required) Data Representations

Currently in the form of Raw Data, Arrays, Fields and Meshes, Each must accommodate different data types

Data I/O Should be able to define and access subsets of data Should be able to define, access, and use subsets of meshes

Data Management Level 1: (Desired but not Required)

“Update” of ghost nodes Should this be level 0? (Rob thinks so) Should this be at the decomposition level or at the data level

Collective operations over fields such as max, min, sup, etc Mesh modification APIs (adaptation, smoothing, flipping) Redistribution (is this the mxn problem?)

Level 0 Interfaces

Raw Data Port Set of smart pointers containing name, location, pointer,

size, type, strides describing a strip of memory collection of 1D buffers each of which can have a different

type Modeled on unix IOvecs Low level and general

can therefore be used as a handle for moving data efficiently

Considered to be local only Array Port

chunk of memory arranged topologically (1d,2d,3d) contains name, dimension, shape should be able to return subsets of raw data can be SPMD

Level 0 Interfaces

Fields and Meshes Fields are defined on mesh extents Fields have a math type (scalar, vector, quaternions,…) Fields can be implemented as Arrays Meshes can be structured or unstructured

Both require nodes with coordinate information and dimension information

Unstructured additionally required elements with connectivity information

Fields and meshes are inherently related (one is no good without the other)

SPMD

Level 0 Interfaces

DataManager Container for named

meshes, fields, arrays and raw data

names must be unique Job is to marshal data (e.g.

to disk or to components needing data)

Data access is via string name

Components that operate on data interact with the DataManager to obtain the data

Multiple managers can be wired to a component

Implies a registry to prevent name conflict

DM1

DM2

Raw Data

Array

Structured Mesh

Unstructured Mesh

CFD

FFT

MxN

VIZT

Design Procedure

To list anticipated operations in the data interface design and to test it against existing applications and mesh field data implementations

DISCUSSION?

OLD STUFF

Relationships

CCAArrays contain raw data extension of raw data in that it contains dmenions and is

decomposable Data Managers can contain fields and meshes, cca

arrays and raw data raw data must be associated with a named entity to be

served up by a data manager Getting subregions of raw data rquired knowing what

object the raw data lives on (how else can you define the subregion)

Interface Ideas from ANL: Background

Existing mesh management package: SUMAA3d Targeted at Finite Element Applications Handles

Mesh Infrastructure Management Mesh Modifications (adaptation, coarsening, smoothing,

flipping) Uses URB partitioning (tightly coupled to mesh data

structures) Only simple geometry and boundary definition

Primary data structures are linked lists of elements, edges, or vertices

User Data can be user-defined via subroutine callbacks; accessed on entity by entity basis

Attempting to define interfaces that decouple a discontinuous Galerkin application from SUMAA3d data structures

Design Philosophy Interfaces are defined in small groups of related

functionality Atomic operations required Additional interfaces for performance or specialized

operations The required interface set must be small to

encourage adoption by mesh management tool providers Not all interfaces need be implemented by all mesh

management packages Defined interfaces for Vertices, Edges, and Elements

Some geometric and topological information Simple data I/O that assumed type double By no means complete - prototype only for testing

performance Original goal is to minimize changes to SUMAA3d

General Mesh Methods

Class MeshComponent {MeshComponent();~MeshComponent();int setOption(char *, int); // also for float, double, charint getOption(char *, int *); // also for float, double, charint publishOptions();int setNumberofVertexUnknowns(int); // also Edge, Elementint createMesh();int intializeUserData(); int getNumberOfVertices(int *); // also Edge, Elementint getMeshVertexList(VertexList *); // also Edge, Element

}

Vertex Methods

Class Vertex {Vertex();~Vertex();int getID(int *);int getVertexCoordinates(double **);int getUserData(VertexUserData *);int setUserData(VertexUserData);int getNextVertex();}

Edge Methods

Class Edge {Edge();~Edge();int getID(int *);int getVertices(Vertex **);int getElements(Element **);int getUserData(EdgeUserData *);int setUserData(EdgeUserData);int getNextEdge(Edge *);

}

Element Methods

Classs Element {Element();~Element();int getID(int *);int getVertices(Vertex **);int getUserData(ElementData *);int setUserData(ElementData);int getNextElement(Element *);

}

ElementData Methods

Class ElementData {ElementData();~ElementData();int getNumData(int *); int setNumData(int);int getData(double **); int setData(double *);int getData(int num, int *index, double **data); int setData(int num, int *index, double *data);}

EdgeData and VertexData are similiar

ElementList Methods

Class ElementList {ElementList();~ElementList();int append(Element);int prepend(Element);int delete(Element);

}

VertexList and EdgeList are similar