U.S. General Services Administration Collaborative Expedition Workshop #74 June 10, 2008 Overcoming I/O Bottlenecks in Full Data Path Processing: Intelligent, Scalable Data Management from Data Ingest to Computation Enabling Access and Discovery Susan Turnbull, GSA, Almadena Chtchelkanova, NSF, and Robert Chadduck, NARA, Richard Spivack, NIST

2 Collaborative Expedition Workshops Purpose: Monthly open workshops to transcend insularity, encourage collaboration and demonstrate promising capabilities emerging from IT research and development Organize around common purpose, larger than any institution, to appreciate potentials and realities Improve quality of dialogue and collaborative prototyping at intergovernmental crossroads Participants, representing many forms of expertise, return to their settings with a larger perspective of the “whole”

3 Collaborative Expedition Workshops Create conducive conditions for “Breakthrough” Innovations Need to Know -> Need to Share -> Build to Share To be Informed (not Overwhelmed) –by the Combined Complexity of our multiple forms of Expertise Communities of Practice Agile Framework for Building Intergovernmental Services Open Collaboration, Open Standards

4 VASA – 1628 In design we either hobble or support people’s natural ability to express forms of expertise.

5 Building Sustainable Stewardship Practices Across Communities Collaborative Expedition Workshops and Collaborative Work Environment ( Co-sponsors: –1. GSA's Intergovernmental Solutions Office –2. Emerging Technology SC (ETSC), Architecture and Infrastructure Committee of the Federal CIO Council – –3. Subcommittee on Networking and Information Technology R & D (NITRD) CGs: including Social, Economic and Workforce Implications of IT and IT Workforce Development (SEW), Human-Computer Interaction-Information Management (HCI-IM) and High End Computing (HEC) –

6 Emerging Technology Subcommittee (ET SC), CIO Council Tuning ET Together From Stovepipes to Wind Chimes Purpose: “Incubator” organizing process to accelerate shared discovery, maturation, and validation of community-based capabilities. Common understanding of scenarios Greater foresight and discernment Improved collaboration Sustainable life-cycles

7 Emerging Technology Subcommittee - ET SC Key FY08 Activities 1.Conduct Collaborative Expedition Workshops with GSA and Subcommittee on Networking for IT Research and Development 2.Conduct ET Life-cycle process – (StratML)StratML ET SC Co-chairs Susan TurnbullRichard Spivack

8 Key FY08 Activities 1. Conduct Collaborative Expedition WorkshopsCollaborative Expedition Workshops Purpose: Monthly open workshops to encourage collaboration among government and community implementers of IT and to demonstrate promising capabilities emerging from IT research that aligns with FEA principles “Facilitate strategic dialogue among communities of interest. Through the Expedition Workshops, sponsored by AIC, interested participants experience and learn about new opportunities to adhere to sound architectural principles and implement shared, service- oriented solutions.” from CIOC Strategic Plan Leadership in virtual collaboration (i.e. Data Reference Model, Geospatial Profile) Key FY08 Activities/Deliverables 1.Organize around business scenarios from ET.gov & IT R&D communities that address CIOC Strategic Plan and Architecture Principles for the US Government.IT R&D communities 2.Organize around CIO requests. ET SC co-chairs SEW CG co-chairs Susan Turnbull, ET SC Susan Turnbull, SEW Richard Spivack, ET SC Suzi Iacono, SEW

9 Key FY08 Activities 2. Conduct Purpose: “Continue to develop more efficient and effective methods for sharing information on emerging technologies.” CIOC Strategic Plan ET.gov stages: 1. Identification: anyone registers ET component using XML schema 2. Subscription: community forms around high potential component 3. Stewardship: community recognized by ET SC (i.e. IPv6, StratML)StratML 4. Graduation: component recognized by Services SC for inclusion in CORE.gov Key FY08 Actions 1.Explore partnering with other federal settings involved in technology evaluation and transfer 2.Conduct Collaborative Expedition workshops to support networking among ET communities Contact Information Jim Disbrow, Program Mgr. Susan Turnbull Spivack,

10 Lessons Learned Connecting the Cultural DOTs - Dialogue, Openness, Transparency 1. Create environment to appreciate the “whole picture” – transcend insularity 2. Practice plausible scenarios “on Purpose” –Monthly public workshops, no fee, supports remote participants (shared screen, chat room) and public archive, including audio files, discussion forum –Assume strategic leadership roles while “thinking out loud together” 3. Shared Purpose is the organizing force in public workshops –Purpose that is larger than any organization – including government; influences structure and participation more than lines of control

11 Lessons Learned – Summary of Purpose Improved Ability to Appreciate the Whole Picture –overcome cultural differences in order to increase returns and decrease risk Improved Ability to Engage in Sustained Dialogue –low-cost, low risk opportunities to dialogue and exchange views on emerging issues, enabling trust and mutual sense of purpose to meet future challenges together Improved Resource Allocation Process for Achieving Results –find common ground and shared understanding across funding, implementation, and accountability processes, to eliminate delays, disincentives, and indecisiveness from non-aligned processes

12 Lessons Learned – Organizing via Communities of Practice Collaborative Space Augments  Solid Past contributions and conversations always available Content never lost, wiki changes visible/ accountable by name High confidence level in 24/7 availability Hosted on high performance infrastructure Platform independent Any file format in shared repository fine-grained access – “virtual pointer on infinite whiteboard“ (persistent identifiers) People’s Natural Ability for Dialogue and Sharing Fluid Augments flow of purposeful conversations Sharing is paramount Context advances understanding Supports quality of dialogue, openness and transparency needed to build trust Supports CoP planning and development of events and documents Uses only everyday tools: phone and browser Open or closed communities Community sets the pace

13 Key Findings Building Sustainable Stewardship Practices Across Communities FY03 - Agile business components not easily discovered by e- government managers resulting in lost opportunities FY04 - Emerging Technologies (web services, grid computing, and semantic web) to tune up Innovation Pipeline with better linkages. FY05 - Collaborative Work Environment (including wiki) expands effective networking across intergovernmental communities FY06-07 – Diverse Communities co-organizing the workshops FY08- Practicing alignment with “real” national scenarios and joint workshops with NITRD Subcommittee Coordinating Groups ( Networking for Multiplicative Returns) –Building shared understanding of fundamental concepts needed for communities representing diverse forms of expertise, to work together to leverage toward improved citizen service delivery at lower cost.

14 Going Forward: From Stovepipes to Wind-Chimes "Frontier Outpost" to open up quality conversations, augmented by “light-weight” tools, to leverage collaborative capacity of united, but diverse sectors of society, seeking to discover, frame, and act on national potentials. 73 workshops since March, participants per workshop, many Communities of Practice Wiki, shared files, discussion forum, chat room, shared screen display FY06: 1.1 million visits to site, 3.88 million file downloads, FY07: 1.7 million visits to site, 5.62 million file downloads FY08 Alignment: Networking for Multiplicative Returns –Building shared understanding of fundamental concepts needed for communities representing diverse forms of expertise, to work together to leverage toward improved citizen service delivery at lower cost.

15 Common Workshop Questions How can multiple Communities of Practice (CoP) organize around common mission needs to build shared understanding in a manner that encourages creativity, trust, agility, and greater value from assets? How can shared understanding around urgent cross-boundary scenarios be accelerated and what is the role of collaborative prototyping? How can maturing, light-weight (Web 2.0) tools support governance and transformational potential of inter-organizational communities and their host institutions? Workshop Questions 2008Workshop Questions 2008

16 Today’s Workshop 8:30am - Check-in and Coffee 8:45am - Workshop Overview - Susan Turnbull, Robert Chadduck, Almadena Chtchelkanova, Richard Spivack 9:00am - Welcome Christopher Greer, Ph.D., Director, The National Coordination Office, Networking and Information Technology Research and Development, The Executive Office of the President 9:15am – Introductions: Attendee self introductions, including brief statements of interests and questions in light of perspectives and practical operational challenges to optimize throughput in processing ultra-large scale data collections - Robert Chadduck, NARA, Moderator

17 Today’s Workshop 10:00am – Panel One: Data Management Approaches Contributing to Optimize I/O in Full Data Path Processing, Robert Chadduck, NARA, Moderator Dr. Michael Folk, Ph.D., Director, The HDF Group (“THG”), HDF5 Experiences with I/O Bottlenecks Louis Reich, NASA Goddard Space Flight Center/ Computer Sciences Corporation, Research Findings Concerning the Utility and Scalability of the XFDU and Related Technologies in the Packaging and Validation of Very Large Digital Information Products Dr. David Du, Ph.D., Program Director, Directorate for Computer and Information Science and Engineering, Division of Computer and Network Systems, The National Science Foundation, Long term End-to-End Security, Privacy, and Provenance 11:30pm – Lunch

18 Today’s Workshop 12:30pm – Panel Two: Mass Storage Systems & Technologies Interests and Research Dr. Reagan Moore, Ph.D, Director, Data Intensive Cyberinfrastructure Environments Groups, The University of California, San Diego, Moderator and Managing Massive Data Collections Michelle Butler, Technical Program Manager, Storage Enabling Technologies Group, The National Center for Supercomputer Applications (“NCSA”) Dr. Ethan Miller, Ph.D, Associate Professor, The Department of Computer Science, The University of California, Santa Cruz (UCSC), Search and Indexing for Petabyte-scale Storage and Beyond Paul Nowoczynski, Advanced Data Management Specialist, The Pittsburgh Supercomputer Center

19 Today’s Workshop 2:00pm – Panel Three: File Systems and I/O Interests and Research Dr. Gary Grider, Ph.D, Los Alamos National Laboratory, Moderator and Highlights of File Systems and I/O Research and Implications for Information Lifecycle Management (ILM) Challenges Dr. Garth Gibson, Ph.D., Carnegie-Mellon University, Data Reliability, Failure, Failure/ Operational Data Release Dr. Henry Newman, Ph.D., Instrumental, Inc., Emerging Role of Standards in Information Lifecycle Management (ILM) Bob Rogers, Chief Technology Officer, Application Matrix, LLC and ISM SNIA working group, Challenges and Opportunities of Information Lifecycle Management (ILM)

20 Today’s Workshop 3:30pm – Open Discussion: Opportunities for Synergy in Next Steps, Including Potential Commonalities in Technologies or Approaches in Response to “Hard Problems” to Optimize Full Data Path Throughput 4:00pm – Poster Sessions Led by University Researchers: HEC-URA FSIO research program, mass storage technologies, data management approaches, etc. Greg Ganger,The Carnegie Mellon University Matthew Wolf, The Georgia Institute of Technology Phil Carns, Argonne National Laboratory Walt Ligon, Clemson University Pete Wyckoff, The Ohio Supercomputer Center Alok Choudhary, Northwestern University Remzi Arpaci-Dusseau, University of Wisconsin Xian-He Sun, The Illinois Institute of Technology

21 Today’s Workshop Tzi-cker Chiueh, The State University of New York at Stony Brook Xiaodong Zhang, The Ohio State University Mahmut Kandemir, Pennsylvania State University Scott Brandt, The University of California, Santa Cruz Paul Nowoczynski & Jared Yanovich; The Pittsburgh Supercomputer Center; Results and Developments in Scalable Lightweight Storage Hierarchies Yong Chen; Department of Computer Science, The Illinois Institute of Technology; Server-Push Architecture for Improving I/O Access Performance Nawab Ali, Department of Computer Science, The Ohio State University; Redesigning Parallel File Systems Using Object- based Storage Devices Xin Li, University of Rochester, Reference-Driven Performance Anomaly Identification 5:30pm or 6:00pm – Adjourn

22 Invitation to Upcoming Expedition Workshops July 15 - Peer Review and Scientific Knowledge Validation August 19 - Broad Public Participation/ Green IT Sept Science of Science and Innovation Policy (being developed with NITRD CGs) Contact Susan.Turnbull at gsa.gov Questions?? Discussion