HPC USER FORUM I/O PANEL April 2009 Roanoke, VA Panel questions: 1 response per question Limit length to 1 slide

Panel Format

Panel Members

Q1. Parallel NFS finally is here! With the formalization of Parallel NFS as a standard - what steps are being provided to enable this to be hosted on current (and future) platform choices? This is really a development question for Garth and Peter Consumers should require this as part of the RFP process Remember that pNFS is a protocol and not a file system POSIX semantics for writing to the same file are not maintained Though pNFS is a very good thing is does not solve world peace and global warming The underlying file system might need to be modified to support the new high speed I/O

Q2. Parallel NFS – implementation details… What tools are available to help optimize this (from application layer level all the way to archival stage)? What is missing and who should provide it? Actually there are no tools for end-to-end integrated I/O optimization and it would require a major effort to provide an integrated tool kit There are many obstacles to these types of tools such as understanding the configuration and contention from other processes Example: A user could make a 30% performance improvement I/O of a job and it might not be recognized given contention from other work How do you measure the I/O of a job without measurement of the system

Q3. Physical media interconnects … We all are facing complexity and cost issues. With IB or 10 GbE (40/100 GbE) : where should the HPC community should focus its resources on - for all I/O? The community should be looking to the future at what will be the commodity technology will be the winner That in my opinion will be FCoE and GbE In my opinion sites should begin to plan for this future by planning for this by looking at NICs and cabling plans The process of change will being in earnest in 2010 and given legacy environments will likely take at least 5 years to complete Things take longer than expected with storage Take the 3.5 to 2.5 disk drive change now in progress for an example

Q4. Layer protocols above the interconnects Too many standards. interconnects, media layers are issues today. iSCSI/FCOE/FcoCEE/FCoIB have all been touted as the solution(s). Is it even relevant in the HPC arena? Is fragmentation the only choice? In my opinion ISCSI will not make it as an HPC solution The issue is encapsulation of metadata and small blocks and wasting channel bandwidth HPC is not all about streaming I/O Small block random happens more than you think and is the bounding factor for many HPC Commodity technology will eventually overtake IB was successful when PCIe became available And it will limit us again 40 GbE will win in cost over QDR Maybe not at first but eventually given volume

Q5. I/O issues not yet addressed? What do you consider to be the top 3 main (technical or human) issues in HPC I/O? 1 HPC file systems have historically had a problem opening a single file for all nodes, which means the file counts are growing significantly 2 There are no really I/O tools for users to tune applications on operational systems especially 3 User data management (ILM) frameworks are not part of the standard process and are implemented in user space Part of the complexity of this is a function of #1