Common Component Architecture for High-Performance Computing CCA Working Group info: Rob Armstrong,

What’s the problem? 9HP computing has as much need for component arch’s as common computing does l Complex and hard to program algorithms, data structures. l Time-to-market is the consideration (e.g. SSI, ASCI curves). 9Currently little reuse of code. l attempts: PetSc, POET, POOMA l most HP applications are written from the ground up for a single purpose.

HP Component Architecture meets a critical DOE need that is not and will not be addressed by Silicon Valley.

The Common Component Architecture Requirements 9Simple l to adopt l to understand l to use 9Support a composition mechanism that does not impede high-performance component interactions. 9Language interoperability l Fortran, C, C++, Java l interface introspection 9Meant to live with and rely on other component frameworks to provide services... l and other complicated, hard-to-implement stuff.

The problem with existing Component Architectures 9Overall: l hi-latency connection is OK for loosely-coupled app’s: network OK for component composition l not OK for high-performance app’s. 9Specifics l JavaBeans: composition only by events/listener. l COM: no real interoperability, not vendor neutral. l CORBA2: no component model. l CORBA3 components: OK but complicated, only for loosely-coupled app’s. 9Seek to supplement existing CA’s with an HP add-on.

Who are we? 9Researchers in the HP components field that are dedicated to forming an open standard for HP components. l addressing the concerns of HPC. 9Originated from DOE’s DOE2K program l has its place in ASCI: Software Integration Curve. l participation from universities.

CCA Active Participants * 9Dennis Gannon (Indiana U) 9Randall Bramley (Cal Tech) 9Andy Cleary, Scott Kohn, Brent Smolinski (Livermore) 9Kate Keahey, Pete Beckman, Bill Humphrey, Pat Fasel, (Los Alamos) 9Rob Armstrong, Robert Clay, Carl Melius, Noel Nachtigal (Sandia) 9Al Geist, Jim Kohl (Oak Ridge) 9Lois Curfman McInnes, Barry Smith, Paul Hovland Brian Toonen (Argonne) 9Steve Parker (U Utah) 9John Wu (Lawrence Berkeley) 9Jarek Nieplocha (Pacific Northwest Labs) * Contact Rob Armstrong if interested in joining the CCA mailing list.

Scientific IDL provides OO interoperability to C, C++, Java, Python, Fortran 77/90 9Follows a Java-like interface model that will “glue” languages together with a minimum performance hit l expected overhead: ~2-3 virtual function calls 9IDL interoperability library supports interface introspection l Necessary for run-time discovery of objects by frameworks l Allows interfaces to be “recognized” by the framework without having specific advanced knowledge.

No plan for world domination... 9do not try to organize complexity top-down. 9Provide a medium of exchange 9believe in Adam Smith’s invisible hand...

What have we come up with? 9Scientific IDL spec. l provides language interoperability only - not part of the component arch. per se. l introspection. 9Metaphor from visual programming (e.g. AVS) l instead of data-flow ports are linked with interfaces l Provides/Uses design pattern l 0.5 CCA specification:

Port model hooks up an interface from one component to another

Ports preserve the high-performance of direct connections plus versatility of distributed object systems 9Allows for directly connected interfaces: the next component is one function call away. 9Adapters will create network-distributed objects out of the same components without altering them.

Generalized ports and the provides/uses design pattern for coupling components Component 1Component 2 CCAServices 2 4 registerUsesPort("A") 1 Port addProvidesPort(,"A") = getPort("A") Port 3

Where are we going? Solving the MxM and MxN problem SC99 reference implementation of CCA with Equation Solver Interface ESI component. experiments with the architecture.

BoneYard

An IDL compiler automatically generates “glue” code for supported languages IDL Compiler IDL Interface Description Stub Glue Code Skeleton Glue Code Caller Language Library Language Language Mappings Compiler Details IDL Database called by library user wraps library implementation IDL (e.g. code written in C)currently C and f77- C++, Java soon

Ports can be used to characterize potential links to other components 9Each CCA Component has l A list of “input” Ports where An input gPort defines an interface to services provided by the component. l A list of “output” Ports where An output gPort is the interface to a set of services required by the component. l Other “self-identification” features allow component containers to probe the component properties. This may be rolled into its own gPort. Client OutputPort Server InputPort

CCA is a component architecture that does not dictate frameworks or runtime 9We are trying to create components that are usable under a variety of frameworks; 9We are not making a new framework implementation. l provide a means for discovering interfaces, similar to IUnknown. l specifically exclude how the components are linked, that is the job of a framework. l provide language-independent means for creating components from existing and future languages.

Where are we? Details of the CCA specification...  Overall:  draft Port spec in RFC stage:  draft IDL spec in RFC stage:  CCA Demo applet & plug-in: 9Multiple test codes already exist. 9Working on repository/naming issues.

(1) Add pea to pod component Pea Pod (2) Create input and output port objects via introspection Port (3) Connect components User- created port Port Programmer can create a new Port by tying methods to a Port

Conclusions 9Philosophy: Not trying to “own the world.” Trying to provide a “coin-of-the-realm” -- a medium of exchange -- that promotes an HPC software economy.  Overall:  Draft Port spec in RFC stage:  Draft IDL spec in RFC stage:  CCA Demo applet & plug-in:

SPMD view Different processes

What is a software framework and components 9Component Concepts l Autonomous interchangeable parts l Links between components that create an application l Builders l Framework provides context or environment to components

Characteristics of Component Arch’s 9Composition Phase 9Run Phase

Language interoperability is a critical first step towards software interoperability 9Software re-use is often hampered by language barriers l DOE labs use many languages (f77, f90, C, C++, Java, Python) l can be difficult for some languages to call others (f77 to C++) 9We are developing IDL technology for interoperability l interface definition language (IDL) describes calling interface l tools automatically generate code to “glue” languages A. Cleary, S. Kohn, S. Smith, B. Smolinski, Language Interoperability Mechanisms for High-Performance Scientific Applications, SIAM Interoperability Conference, C C++ f77 f90 Python Java IDL

CCA Ports employ a Uses/Provides design pattern for connections One component uses an interface provided by another. At the containing framework’s option: Allows direct connections. Allows connections through remote proxies all with the same components

Where we are going... 9Port and IDL spec’s need to be solidified l time and testing are now the big contributors to the spec. l Publication: Web, HPDC in Aug, San Diego, demo SC99. 9CCA as a standards body l soliciting, accepting/rejecting proposals of interfaces and implementations for inclusion into the core CCA.

Command-line applet that implements the specification 9link together pretend components to do string manipulations 9Source code available

9Component - Encapsulated software object that provides a certain functionality or service and can be used in conjunction with other components to build applications A component consists of l an API (abstract interface) and l one or more component implementations l common behavior among these components 9CCA uses a peer component model - No particular component assumes it is “in charge” of the others By definition components are software modules that are composable...

ESI is a multi-lab effort writing an equation- solver component specification DOE initiative, building on previous efforts. SNL Ben Allan, Robert Clay, Lee Taylor, Alan Williams LLNL Colin Aro, Andy Cleary, Rob Falgout, Juliana Hsu LANL Mike DeLong, Robert Ferrell, Wayne Joubert ANL Barry Smith, Lois Curfman McInnes ORNL Noel Nachtigal, Jack Dongarra LBL John Wu CSU Chico Kyran Mish U.Indiana Randy Bramley 9technical forum: ( )

Equation Solver Interface (ESI) group is defining interfaces for solvers

The goal of CCA group is enable applications constructed entirely of components Discretization Algebraic SolversParallel I/O Grids Data Reduction Physics Modules Optimization Derivative Computation Collaboration DiagnosticsSteeringVisualization Adaptive Solution

gPorts connect component interfaces in a data-flow-like way 9Needs to work for tightly coupled and dist. objects l tightly coupled for performance allow overhead approx. a virtual function call l transparently marshal args and return values transparency over various networks and environments 9gPorts is intended to provide a more efficient and scalable component connection mechanism l Input/Output gPorts can call each other explicitly and directly. 9Recent (7 Nov ‘98) CORBA3.0 spec hints at similar “port” concept.

Where we are now...  draft gPort spec in pre-RFC stage:  draft IDL spec in pre-RFC stage: 9Multiple test codes already exist. l experiments with the Common Component Architecture. 9Working on repository/naming issues. l Most likely to be developed from LDAP or JNDI.

Where we are going... 9gPort and IDL spec’s need to be solidified l a formal RFC must be accomplished for both IDL and gPorts. l will be tested in the working group’s own frameworks and refined from real-world experiences. l the work will be published and advertised in appropriate places. 9Repository/Naming Issues Addressed l Make appropriate hooks into the spec. to import a “standard” registry/repository/naming scheme.

9 CCA is our only hope l Here is the list of pan-lab HP software integration groups: ESI - Equation Solver Interface specific only to this one field CCA 9 The outside world is componentizing and integrating software, we cannot remain in the one-lab-one-code mode forever. l DOE has special needs with regard to HP software that the outside world is not addressing. Conclusion

(1) Add pea to pod component Pea Pod (2) Create input and output port objects via introspection Port (3) Connect components User- created port Port Programmer can create a new Port by tying methods to a port

CCA Plans: A Big Picture Vector component Linear solver component Non-linear solver component Multi-component interoperability Rob: Paul’s from Barry Rob: Paul’s from Barry

IDL specification for the SMG structured multigrid preconditioner from hypre package hypre { class stencil { static stencil NewStencil(in int dim, in int size); int SetStencilElement(in int index, in array offset); }; class grid { static grid NewGrid(in mpi_com com, in int dimension); int SetGridExtents(inout array lower, array upper); }; class vector { static vector NewVector(in mpi_com, in grid g, in stencil s); int SetVectorBoxValues(/* long argument list omitted */); }; class matrix { /* matrix member functions omitted */ }; class smg_solver { int Setup(inout matrix A, inout vector b, inout vector x); int Solve(inout matrix A, inout vector b, inout vector x); }; );

Common Component Architecture Working Group Rob Armstrong

What we are doing 9The problem l Many incompatible frameworks. l Much replication of s/w effort. l Brave new world of network grids presents many opportunities & challenges to interoperability. 9The solution l A component arch. enabling interoperability between modules for scientific computing. l Not defining a new framework, but a standard that will work within many frameworks.

Who we are 9Participants design and implement components/frameworks for their respective institutions. l know component arch’s and HP computing meeting DOE lab needs. 9Venue: Meet every 3 months.

The goal of CCA group is enable applications constructed entirely of components Discretization Algebraic SolversParallel I/O Grids Data Reduction Physics Modules Optimization Derivative Computation Collaboration DiagnosticsSteeringVisualization Adaptive Solution

Where we are now...  draft Port spec:  draft IDL spec: 9Multiple test codes already exist. Command-line demo: 9Working on repository/naming issues. l Will use the existing Ports mechanism to access these and other services.

Scientific IDL provides OO interoperability to C, C++, Java, Python, Fortran 77/90 9Follows a Java-like interface model that will “glue” languages together with a minimum performance hit l expected overhead: ~ 2-3 virtual function calls 9IDL interoperability library supports interface reflection l Necessary for run-time discovery of objects by frameworks l Allows interfaces to be “recognized” by the framework without having specific advanced knowledge of active components

CCA Ports employ a Uses/Provides design pattern for connections One component uses an interface provided by another. At the containing framework’s option: Allows direct connections. Allows connections through remote proxies all with the same components

Where we are going... 9Port and IDL spec’s need to be solidified l time and testing are now the big contributors to the spec. l Publication: Web, HPDC in Aug, San Diego, demo SC99. 9CCA as a standards body l soliciting, accepting/rejecting proposals of interfaces and implementations for inclusion into the core CCA.

CCA Active Participants * 9Rob Armstrong, Robert Clay, Carl Melius, Bill Mason (Sandia) 9Dennis Gannon (Indiana U) 9Randall Bramley (Cal Tech) 9Andy Cleary, Scott Kohn, Brent Smolinski (Livermore) 9Kate Keahey, Pete Beckman, Bill Humphrey, Pat Fasel, (Los Alamos) 9Al Geist, Noel Nachtigal (Oak Ridge) 9Satish Balay, Lois Curfman McInnes, Barry Smith, Paul Hovland, Lori Freitag (Argonne) 9Steve Parker (U Utah) 9Brent Milne (Lawrence Berkeley) 9Jarek Nieplocha (Pacific Northwest Labs) * Contact Rob Armstrong if interested in joining the CCA mailing list.

gPort model hooks up an interface from one component to another

User applications can now invoke hypre routines from both C and Fortran 77 integer b, x integer A integer smg_solver b = hypre_vector_NewVector(com, grid, stencil)... x = hypre_vector_NewVector(com, grid, stencil)... A = hypre_matrix_NewMatrix(com, grid, stencil)... smg_solver = hypre_smg_solver_new() call hypre_smg_solver_SetMaxIter(smg_solver, 10) call hypre_smg_solver_Solve(smg_solver, A, b, x) call hypre_smg_solver_Finalize(smg_solver) hypre_vector b, x; hypre_matrix A; hypre_smg_solver smg_solver; b = hypre_vector_NewVector(com, grid, stencil);... x = hypre_vector_NewVector(com, grid, stencil);... A = hypre_matrix_NewMatrix(com, grid, stencil);... smg_solver = hypre_smg_solver_new(); hypre_smg_solver_SetMaxIter(smg_solver, 10); hypre_smg_solver_Solve(smg_solver, &A, &b, &x); hypre_smg_solver_Finalize(smg_solver); C Test CodeFortran 77 Test Code The end user is completely unaware that IDL tools were used