Parallel I/O A. Patra MAE 609/CE667...

What is Parallel I/O ? zParallel processes need parallel input/output zIdeal: Processor consuming/producing data reads/writes it directly zNot practical for large numbers of processors...

What is Parallel I/O ?

zR. Lusk : yMultiple Processes Participate in the I/O yApplication level parallelism y“File” is stored on multiple disks on a parallel file system yAdditional Interfaces for I/O

What is Parallel I/O ? yFile is stored on multiple disks on a parallel file system

What is Parallel I/O ? yI/O should be parallel at both ends xApplication program end -- with access to single logical file that is distributed across physical disks xI/O should be physically parallel so that parallel performance scales with no. of processors etc.

Parallel File Systems zProvide users with ya consistent name space across the machine, xaids programmers in accessing file data on multiple nodes yphysical distribution of data across disks and network entities, and, xeliminates bottlenecks both at the disk interface and the network, providing more effective bandwidth to the I/O resources

Parallel File Systems zExample Systems yPFS -- Intel Paragon yXFS -- SGI Origin yPIOFS-- IBM SP yPVFS -- Linux cluster

Parallel Virtual File System (PVFS) PVFS system consists of three components: the manager daemon, which runs on a single node, handles permission checking for file creation, open, close, and remove operations the I/O daemons, one of which runs on each I/O nodes, and handle all file I/O application library, through which applications communicate with the PVFS daemons.

Parallel Virtual File System (PVFS)  File striping  File partitioning  Application-oriented interfaces  Operation with existing binaries When using PVFS, nodes who perform computation (compute nodes) must communicate with nodes who perform I/O operations (I/O nodes) in order for file system operations to take place. Application tasks use one of the interfaces available in the PVFS libraries to communicate with the I/O daemons, who use UNIX read() and write() calls to perform I/O operations on the local disks

PVFS zsmall data transfers tend to lead to very poor throughput. zStreams-based approach to data transfer is an attempt to improve overall network throughput by:  reducing the number of control messages  removing stripe and partition dependence on message sizes

PVFS I/O stream between an application and an I/O node resulting from a strided request. Each side calculates the intersection of physical stripe and the strided request. The data is always passed in ascending byte order and is packed into TCP packets by the underlying networking software.

PIOFS -- IBM An RS/6000 SP with several client nodes accessing data at server nodes. The Parallel I/O File System supports simultaneous access of server nodes by multiple client nodes.

PIOFS -- IBM zPIOFS lets you create files as large as 128 Terabytes that span multiple server nodes. zWith PIOFS file partitioning, you can parallelize access to your data without the inconvenience and administrative overhead of maintaining multiple data files. z PIOFS files can be dynamically partitioned into subfiles many different ways, all without altering or moving the contents of the file. z PIOFS supports parallelism in two complementary ways: physically and logically:  A file can be divided physically over multiple disks and servers.  A file can be divided logically into multiple subfiles.

File Partitioning file with numbers File split into 8 subfiles by columns

File Partitioning zFile split into 8 subfiles by rows zFile split in 3 with wrapping

Matrix-Matrix Multiply Tasks 0 and 1 process the first N/2 rows of matrix A Tasks 2 and 3 process the last N/2 rows of matrix A Tasks 0 and 2 process the first N/2 columns of matrix B Tasks 1 and 3 process the last N/2 columns of matrix B

Parallel I/O is active research Only 1 complete MPI-IO implementation available Picture will stabilize over the next few years (1-2)