Towards World-Wide Open T-System PSI RAS, 2006

Long-term Goal  Present a simple in use, automatic dynamic parallelization technology for a scientists, who prefer Windows-based solutions;  Achieve the same performance of the T- applications running on Windows clusters as in a case of the Linux-based clusters;  Advance the Windows version of the Open T-System by adding visualization, fault- tolerance and ability to run T-applications on heterogeneous (Windows+Linux) metaclusters.

Cumputists Wishes  Many scientists prefer to use Windows, not Linux for high-performance computing (by obvious reason)  Many scientists are using C/C++ for high- performance application development  Most scientists are concentrating on the domain-specific problems and do not like like study any kind of MPI-programming technology at all !

Current T-system “community”  PSI RAS (Research & Education)  MSU (Computational Math., Mechanic, Chemistry faculties)  NIICS (satellite image processing)  ChelGU (conformational analysis)  MSIU (education)

Fulfilling Computists Needs  Q: What is a main disadvantage of the Open T-System ?  A: It does not run under Windows (most popular answer at Intel winter school 2005 taking place in Nizhniy Novgorod)

Development priorities 2006  Run T-applications on Windows hosts & Windows clusters (32bit & 64bit modes)  Run T-applications using variants of communications using multiply pluggable DMPI drivers for SAN, LAN, WAN  Make T-system model more clean for T- application developers by computational process visualization  Run T-applications in fault-tolerant mode  Run T-applications on Itanium & PowerPC

Help for application tuning  Develop a tool for T-program computational process visualization (in particular, to present parallelization dynamic in real time)  Built-in lightweight performance analyzer to automate process of T-program optimization

Achieving power of metaclusters  Fault-tolerant computation mode (using advanced re-computation model)  Run Windows T-application on both Windows and Linux clusters (using the wine emulator)  Dynamic enter/leave of computational nodes/segments

MacroScheduler 2006 advance  Reducing “resource exchange” message count at large clusters  Highly efficient algorithm adopted for non- trivial cluster topologies

Current DMPI features  Implements most popular subset of MPI  mpirun/mpiexec type auto-detection  Dynamic loading for appropriate MPI driver  Ability of static MPI binding  Asynchronous operations support  Large number of supported MPI & PVM implementations

Additional DMPI 2006 features  Direct TCP/IP support (able to operate without MPI library)  Multiply MPI drivers working in parallel to mix different transport systems: SCALI, Myrinet, Infiniband, TCP/IP, TCP/IP over HTTP.  MPI Enter/Leave extensions (in addition to standard mpirun/mpiexec running mode)  MPI_Wait & T-Fun waiting coexistence  Automatic fault detection & recovering  Configuration (topology) information

Cross-platform OpenTS  Unified OpenTS source code tree for ALL supported platforms  Unified DMPI source code tree for ALL supported platforms  Extracting all platform-depended features into separate Platform Abstraction Layer (PAL)

OpenTS for Windows installer  Unified installer for Windows XP/2003/MSCCS editions  Installed OpenTS does not require any commercial package (it is enough to download Visual C Express edition for T-program development freely available from site)  Integrated with Visual Studio (T-projects)  Self-testing during installation procedure!  OpenTS SDK is available for T-microkernel developers

Open T-Grid 2006  Cross-platform: run T-applications on Windows and Linux cluster & hosts at the same time (using WinAPI simulation on Linux)  Fault-tolerance: using new DMPI facilities for immediate recovering and unique functional nature of T-system  User-friendly: run T-visualization as an screensaver or as widget/applet in the taskbar  Safety: digital signature check support for T-executables

Open T-Grid 2006  Cooperative usage of the Windows and Linux clusters and single hosts  World-Wide T-system-based computational platform  Good (and awaited) platform for scientists from Universities

Official T-System projects (2006) (donation-ordered)  MSCCS (Microsoft RUS ~ $60.000)  T-GRID (RAS ~ $50.000)  T++ & T# (RAS ~ $30.000)  TRIADA (Russia-Belorussia ~ $40.000)

Cost Analysis  Most new OpenTS features are developed for POSIX environment at the current time.  To adopt presented features for Windows we need reasonable financial support at the next year

OpenTS 2006 restrictions scheduled to be removed  Restricted C++ OOP support: no tfun class methods, no tfun templates;  Intermediate levels of OpenTS API are open, but not documented yet (Active messages, Mobile objects, Supermemory)  Further performance enhancements (intermediate levels template specialization)