Presentation is loading. Please wait.

Presentation is loading. Please wait.

Priority Research Direction (I/O Models, Abstractions and Software) Key challenges What will you do to address the challenges? – Develop newer I/O models.

Published byKeeley Waggoner Modified over 9 years ago

Similar presentations

Presentation on theme: "Priority Research Direction (I/O Models, Abstractions and Software) Key challenges What will you do to address the challenges? – Develop newer I/O models."— Presentation transcript:

1 Priority Research Direction (I/O Models, Abstractions and Software) Key challenges What will you do to address the challenges? – Develop newer I/O models and higher level abstractions (datasets based techniques that exploit specialized applications) – Purpose-driven and customizable I/O (e.g., checkpointing, analytics, external communication (workflow) – Incorporate I/O into programming models and languages – Utilize I/O delegation for offloading I/O within user space, caching, data reorganization – Integrate online analytics and data management Programming and Abstraction : how is I/O viewed from 1M+ processes? The file I/O abstraction is not good enough nor scalable. Make I/O independent of number of processes with predictable performance What capabilities will result? - Higher-level abstraction (e.g., datasets, specialized data management) -Purpose-driven I/O (e.g., checkpointing, analytics, external communication in a workflow) - Customizable I/O -I/O Delegation and Active Storage with I/O and processing as a service How will this impact the range of applications that may benefit from exascale systems? -More control and significantly reduced complexity in I/O (3-5 years) -Portability of application WRT I/O (3-5 years) -Predictable performance (5+ years) -Maximize use of data while available (3-5) -Real-time Knowledge Discovery and Insights (10+ years) Summary of research direction Potential impact on software component Potential impact on usability, capability, and breadth of community

2 4.x I/O Models, Abstractions and Software Technology drivers – File systems with traditional semantics are not scalable – I/O architectures as an independent and separate component does not scale Alternative R&D strategies – Extend current file systems – Develop newer layers on top of current file systems – Develop newer I/O models and higher level abstractions (datasets based techniques that exploit applications domains) – Purpose-driven and customizable I/O (e.g., checkpointing, analytics, external communication (workflow) – Develop techniques to concurrently exploit the data and perform analytics when it is created; that is, embed online analytics – Incorporate I/O into programming models and languages – Use databases – I/O Delegation and Active Storage with I/O and processing as a service Recommended research agenda – Develop newer I/O models and higher level abstractions (datasets based techniques that exploit specialized applications) – Purpose-driven and customizable I/O (e.g., checkpointing, analytics, external communication (workflow) – Incorporate I/O into programming models and languages – Active Storage with I/O and processing as a service – Utilize I/O delegation for offloading I/O within user space, caching, data reorganization etc. – Develop techniques to concurrently exploit the data and perform analytics when it is created; that is, Integration of data analytics, online analysis and data management Crosscutting considerations – Programming models and languages – Architectures

3 Priority Research Direction (Newer Storage Devices (SCM/SSD) and I/O Hierarchies) Key challenges What will you do to address the challenges? -Develop balanced architectures with newer devices embedded within the system -Develop new I/O models, software, runtime systems and libraries to exploit these hierarchies -Develop new file systems or special-purpose data management layers -Intelligent and proactive caching mechanisms Brief overview of the barriers and gaps Performance, energy footprint and scalability of current storage devices is limiting -Incorporation of newer storage devices such as SCM, SSD -Optimizations for managing newer hierarchies What capabilities will result? -Orders of magnitude faster I/O and performance - Significant potential for power optimizations in the I/O subsystem What new methods and components will be developed? - Software layers for managing newer devices and memory hierarchy How will this impact the range of applications that may benefit from exascale systems?* -Much faster I/O and highly optimized sustained performance (3 years) -Significant reduction in the cost of checkpointing (3 years) -Real-time knowledge discovery and insights (6+ years) -Much simpler data management (5 years) * This timeline is relative to the time thee devices are incorporated into the architectures Summary of research direction Potential impact on software component Potential impact on usability, capability, and breadth of community

4 4.x Newer Devices and Hierarchies Technology drivers – Disks based storage systems not scalable – Newer Storage devices such as SCM and SSD provide a potential to significantly improve performance and reduce power consumption by orders of magnitude Alternative R&D strategies – Build balanced architectures with newer devices embedded within the system – Develop new I/O models, software, runtime systems and libraries to exploit these hierarchies – Develop new file systems or special-purpose data management layers – O/S manages the new memory hierarchy (for I/O purposes) – Intelligent and proactive caching mechanisms Recommended research agenda – Develop balanced architectures with newer devices embedded within the system – Develop new I/O models, software, runtime systems and libraries to exploit these hierarchies – Develop new file systems or special-purpose data management layers – Intelligent and proactive caching mechanisms Crosscutting considerations – Power optimizations – Potential to significantly enhance resiliency – Architectures – Operating System

5 4.x Technology drivers – Data movement from/to systems is sequential (single node based) even with multiple streams – Protocol conversion Alternative R&D strategies – Develop parallel data movement software and tools – Special purpose network protocols for parallelism – Scalable Scheduling – Integration of external networks with local file systems Recommended research agenda – Develop parallel data movement software and tools – Special purpose network protocols for parallelism – Scalable Scheduling – Integration of external networks with local file systems Crosscutting considerations – Scheduler

6 I/O, Storage and Data Management I/O delegation I/O Runtime systems for SCM/SSD devices, Newer I/O abstractions Purpose driven I/O and Active Storage, Integration of Analytics and I/O Power optimized, Customizable I/O Integrated with newer Programming Models and Languages SDM for Peta/Exa-bytes Real-time Knowledge Discovery and Insights 2010201120122013201420152016201720182019 Accelerated Scientific insights from Petabytes of Data

Download ppt "Priority Research Direction (I/O Models, Abstractions and Software) Key challenges What will you do to address the challenges? – Develop newer I/O models."

Similar presentations

Technology Drivers Traditional HPC application drivers – OS noise, resource monitoring and management, memory footprint – Complexity of resources to be.

Technology Drivers Traditional HPC application drivers – OS noise, resource monitoring and management, memory footprint – Complexity of resources to be.
1 A GPU Accelerated Storage System NetSysLab The University of British Columbia Abdullah Gharaibeh with: Samer Al-Kiswany Sathish Gopalakrishnan Matei.

1 A GPU Accelerated Storage System NetSysLab The University of British Columbia Abdullah Gharaibeh with: Samer Al-Kiswany Sathish Gopalakrishnan Matei.
Priority Research Direction Key challenges General Evaluation of current algorithms Evaluation of use of algorithms in Applications Application of “standard”

Priority Research Direction Key challenges General Evaluation of current algorithms Evaluation of use of algorithms in Applications Application of “standard”
Priority Research Direction: Portable de facto standard software frameworks Key challenges Establish forums for multi-institutional discussions. Define.

Priority Research Direction: Portable de facto standard software frameworks Key challenges Establish forums for multi-institutional discussions. Define.
Sensor Network Platforms and Tools

Sensor Network Platforms and Tools
4.1.5 System Management Background What is in System Management Resource control and scheduling Booting, reconfiguration, defining limits for resource.

4.1.5 System Management Background What is in System Management Resource control and scheduling Booting, reconfiguration, defining limits for resource.
Introduction CSCI 444/544 Operating Systems Fall 2008.

Introduction CSCI 444/544 Operating Systems Fall 2008.
1 Lawrence Livermore National Laboratory By Chunhua (Leo) Liao, Stephen Guzik, Dan Quinlan A node-level programming model framework for exascale computing*

1 Lawrence Livermore National Laboratory By Chunhua (Leo) Liao, Stephen Guzik, Dan Quinlan A node-level programming model framework for exascale computing*
Adding scalability to legacy PHP web applications Overview Mario A. Valdez-Ramirez.

Adding scalability to legacy PHP web applications Overview Mario A. Valdez-Ramirez.
Introduction to Operating Systems CS-2301 B-term 20081 Introduction to Operating Systems CS-2301, System Programming for Non-majors (Slides include materials.

Introduction to Operating Systems CS-2301 B-term Introduction to Operating Systems CS-2301, System Programming for Non-majors (Slides include materials.
INTRODUCTION OS/2 was initially designed to extend the capabilities of DOS by IBM and Microsoft Corporations. To create a single industry-standard operating.

INTRODUCTION OS/2 was initially designed to extend the capabilities of DOS by IBM and Microsoft Corporations. To create a single industry-standard operating.
Introduction to Systems Architecture Kieran Mathieson.

Introduction to Systems Architecture Kieran Mathieson.
1 Dr. Frederica Darema Senior Science and Technology Advisor NSF Future Parallel Computing Systems – what to remember from the past RAMP Workshop FCRC.

1 Dr. Frederica Darema Senior Science and Technology Advisor NSF Future Parallel Computing Systems – what to remember from the past RAMP Workshop FCRC.
Chapter 13 Embedded Systems

Chapter 13 Embedded Systems
Introduction to Databases Transparencies

Introduction to Databases Transparencies
State Machines Timing Computer Bus Computer Performance Instruction Set Architectures RISC / CISC Machines.

State Machines Timing Computer Bus Computer Performance Instruction Set Architectures RISC / CISC Machines.
Figure 1.1 Interaction between applications and the operating system.

Figure 1.1 Interaction between applications and the operating system.
Introduction to the new mainframe: Large-Scale Commercial Computing © Copyright IBM Corp., 2006. All rights reserved. Chapter 6: Accessing large amount.

Introduction to the new mainframe: Large-Scale Commercial Computing © Copyright IBM Corp., All rights reserved. Chapter 6: Accessing large amount.
Introduction. Why Study OS? Understand model of operation –Easier to see how to use the system –Enables you to write efficient code Learn to design an.

Introduction. Why Study OS? Understand model of operation –Easier to see how to use the system –Enables you to write efficient code Learn to design an.
NPACI: National Partnership for Advanced Computational Infrastructure August 17-21, 1998 NPACI Parallel Computing Institute 1 Cluster Archtectures and.

NPACI: National Partnership for Advanced Computational Infrastructure August 17-21, 1998 NPACI Parallel Computing Institute 1 Cluster Archtectures and.

Similar presentations

Ads by Google