Problem-Solving Environments: The Next Level in Software Integration David W. Walker Cardiff University

31/8/99 David W. Walker, Cardiff University 2 Objectives of this talk n To review the purpose and scope of PSEs n To discuss the requirements for constructing PSEs n To identify the major technologies that will serve as the infrastructure of PSEs n To review the PSEs currently in development and in use

31/8/99 David W. Walker, Cardiff University 3 Why PSEs? n Need: enhanced scientific insight; reduced development costs; improved product quality and industrial efficiency. n Need: transparent means of integrating distributed computers, instruments, sensors, and people. n Need: improved software productivity to extract maximum benefit from advances in computers, networks, and algorithms.

31/8/99 David W. Walker, Cardiff University 4 Why Now? n Confluence of complementary technologies. n Faster networks and communications. n Network software technologies such as CORBA, Java, and XML. n “Big Science” is inherently distributed and collaborative, and needs to migrate to WAN environments to progress.

31/8/99 David W. Walker, Cardiff University 5 What’s the Problem? n High-level, problem-specification languages, often coupled with expert system. For example, PDE solvers, numerical integration, etc. n Problem composition in form of dataflow graph using a GUI. Typically used in modelling and simulation of physical systems.

31/8/99 David W. Walker, Cardiff University 6 PSE Requirements n Expert assistance in problem specification and input. n Transparent access to distributed heterogeneous resources. n Interactivity and computational steering. n Advanced/immersive visualisation. n Integration with other knowledge repositories and databases.

31/8/99 David W. Walker, Cardiff University 7 Technologies for PSEs Hardware: n Increasingly powerful computers n Increasingly fast networks - gigabit ethernet, vBNS, etc. n Immersive visualisation platforms - CAVEs, ImmersaDesks, etc.

31/8/99 David W. Walker, Cardiff University 8 Technologies for PSEs Software: n CORBA for transparent interaction between distributed resources. n Java for platform-independent programming. n XML interface specification. n MPI for message-passing in SPMD codes.

31/8/99 David W. Walker, Cardiff University 9 An Example PSE Architecture Main PSE sub-systems are: n Visual Program Composition Environment (VPCE) for graphically composing applications. n Intelligent Resource Management System (IRMS) for scheduling applications on distributed resources.

31/8/99 David W. Walker, Cardiff University 10 VPCE Overview n GUI is used to build an application from software components - either a java or CORBA object with its interface specified in XML. n Each component may have a performance model and help file. n An annotated dataflow graph is produced that is passed to the IRMS.

31/8/99 David W. Walker, Cardiff University 11 IRMS Overview n IRMS locates software and hardware resources through information servers. n IRMS then schedules components on appropriate resources based on performance models and database of experience from previous runs. Genetic and neural network algorithms may be used.

31/8/99 David W. Walker, Cardiff University 12 The PSE Research Community n European Research Conference on PSEs took place June 1999 in Spain. Next one in summer n EuroTools SIG on PSEs. n Cardiff PSE project web site.

31/8/99 David W. Walker, Cardiff University 13 US Software Infrastructure n Globus: provides core services for grid- enabled computing. n Legion: an object-based metacomputing project. The Grid is a computational and network infrastructure providing pervasive, uniform, and reliable access to distributed resources.

31/8/99 David W. Walker, Cardiff University 14 European Software Infrastructure n UNICORE: Uniform access to Computing Resources. Aimed at providing uniform, secure, batch access to distributed resources. n POLDER: a more ambitious metacomputing project.

31/8/99 David W. Walker, Cardiff University 15 European Software Infrastructure n CODINE: resource management system targeted at optimal use of all software and hardware resources in a heterogeneous networked environment. n CCS: Computing Centre Software - resource management for networked high-performance computers.

31/8/99 David W. Walker, Cardiff University 16 European Software Infrastructure n GRD: Global Resource Director for distributed environments featuring policy management and dynamic scheduling. n NWIRE: Netwide resources - management system for WAN-based resources.

31/8/99 David W. Walker, Cardiff University 17 COVISE Visualisation Environment n The Collaborative Visualisation and Simulation Environment is a distributed software environment that seamlessly integrates simulations, post-processing, and visualisation. n COVISE supports collaborative working, and is available commercially.

31/8/99 David W. Walker, Cardiff University 18 Ctadel and PDE Problems n Code-generation tool for applications based on differential equations using high-level language specifications is an environment for the automatic generation of efficient Fortran or HPF programs for PDE-based problems. n Used in HIRLAM numerical weather forecast system.

31/8/99 David W. Walker, Cardiff University 19 An Environment for Cellular Automata n CAMEL n CAMEL is a CA environment designed for message-passing parallel computers. It hides parallelism issues from a user. CARPET User specifies only the transition function of a single cell of the system with CARPET, a high-level cellular language.

31/8/99 David W. Walker, Cardiff University 20 A PSE for Numerical General Relativity n CACTUS is a collaborative software environment for composing applications for the solution of general relativity problems. n Has been used in distributed computing experiments using Globus. n Interactive visualisation important.

31/8/99 David W. Walker, Cardiff University 21 JACO3: Industrial Design PSE n Java and CORBA based collaborative environment for coupled simulations. n A CORBA based high performance distributed computing environment for coupling simulation codes. n Optimal design of complex and expensive products like airplanes, satellites, or cars.

31/8/99 David W. Walker, Cardiff University 22 A PSE for Stochastic Analysis n Promenvir: Probabilistic mechanical design environment - a metacomputing tool for stochastic analysis. n It can automatically generate a series of stochastic computational experiments, and run them on the available resources n It has been used for optimal design problems in the automobile industry.

31/8/99 David W. Walker, Cardiff University 23 PSE for Engineering Simulations n JULIUS: Joint Industrial Interface for End-User Simulations. n Integrated HPC environment for multi- disciplinary engineering simulations. n Aimed at reducing design time for industrial products. n End-users are engineers.

31/8/99 David W. Walker, Cardiff University 24 Summary n There is an active body of PSE researchers and developers in Europe. n PSEs are used in science, engineering, finance, and manufacturing. n Current emphasis is on PSE infrastructure and prototypes.

31/8/99 David W. Walker, Cardiff University 25 Future Challenges n Maintaining good, reliable performance in distributed environments important. n Need to integrate third party software. n Need visualisation environments that scale from PC up to immersive systems. n Needs standards for interfaces and interaction between PSEs.