Trust & File Systems Rick Wagner HPC Systems Manager.

Slides:



Advertisements
Similar presentations
Desktop and Application Virtualization OHECC 2009.
Advertisements

IT Research Support: Successes & Challenges Martin Siegert Steve Cundy Todd Zimmerman.
Ixonos Plc Marko “Narsu” Rintamäki Senior Test Engineer NEST-IX V1.1 Project platform in Test Management.
OpenCloud: Value-Add Cloud on Internet2 Larry Peterson Open Networking Lab.
ASCR Data Science Centers Infrastructure Demonstration S. Canon, N. Desai, M. Ernst, K. Kleese-Van Dam, G. Shipman, B. Tierney.
Serving the Research Mission: An Approach to Central IT’s Role Matthew Stock University at Buffalo.
Rodger Hess ScienceDMZ Switch Multiple 10 GE connections to CENIC’s HPR Network OpenFlow Including GENI Support Performance Measurement.
UNCLASSIFIED: LA-UR Data Infrastructure for Massive Scientific Visualization and Analysis James Ahrens & Christopher Mitchell Los Alamos National.
Roy Hockett April 28, The science is the key to the proposal, without the science the proposal has little chance. What infrastructure.
Building a Cluster Support Service Implementation of the SCS Program UC Computing Services Conference Gary Jung SCS Project Manager
Legion Worldwide virtual computer. About Legion Made in University of Virginia Object-based metasystems software project middleware that connects computer.
What is it? Hierarchical storage software developed in collaboration with five US department of Energy Labs since 1992 Allows storage management of 100s.
Dell IT Innovation Labs in the Cloud “The power to do more!” Andrew Underwood – Manager, HPC & Research Computing APJ Solutions Engineering Team.
“Introduction to UC San Diego’s Integrated Digital Infrastructure” Opening Talk IDI Showcase 2015 University of California, San Diego May 6-7, 2015 Dr.
FACULTY OF COMPUTER SCIENCE OUTPUT DD  annual event from students for students with contact to industry (~800 visitors)  live demonstrations  research.
Approaching a Data Center Guy Almes meetings — Tempe 5 February 2007.
Shared Ownership of Change Rob Super. Shared Ownership Across work streams Across Campuses Between central business process owners and distributed end.
Foundation Pathways in Technology BEng and BSc Programmes in Engineering, Computing and Mathematics with Integrated Foundation Year.
1 Building National Cyberinfrastructure Alan Blatecky Office of Cyberinfrastructure EPSCoR Meeting May 21,
3DAPAS/ECMLS panel Dynamic Distributed Data Intensive Analysis Environments for Life Sciences: June San Jose Geoffrey Fox, Shantenu Jha, Dan Katz,
DuraCloud Managing durable data in the cloud Michele Kimpton, Director DuraSpace.
1 Advanced Storage Technologies for High Performance Computing Sorin, Faibish EMC NAS Senior Technologist IDC HPC User Forum, April 14-16, Norfolk, VA.
Research & Academic Computing Bradley C. Wheeler Associate Vice President & Dean.
Gurcharan S. Khanna Director of Research Computing RIT
1 Developing a Data Management Plan C&IT Resources for Data Storage and Data Security Patrick Gossman Deputy CIO for Research January 16, 2014.
1. 2 Corollary 3 System Overview Second Key Idea: Specialization Think GoogleFS.
Bio-IT World Asia, June 7, 2012 High Performance Data Management and Computational Architectures for Genomics Research at National and International Scales.
Data Science at Digital Science October Geoffrey Fox Judy Qiu
UNM RESEARCH NETWORKS Steve Perry CCNP, CCDP, CCNP-V, CCNP-S, CCNP-SP, CCAI, CMNA, CNSS 4013 Director of Networks.
© 2010 Voltaire Inc. HPCFS AT ORLANDO LUG 2011 BILL BOAS PATH FORWARD FOR LUSTRE COMMUNITY System Fabric Works.
HPCVL High Performance Computing Virtual Laboratory Founded 1998 as a joint HPC lab between –Carleton U. (Comp. Sci.) –Queen’s U. (Engineering) –U. of.
Looking Ahead: A New PSU Research Cloud Architecture Chuck Gilbert - Systems Architect and Systems Team Lead Research CI Coordinating Committee Meeting.
14 Aug 08DOE Review John Huth ATLAS Computing at Harvard John Huth.
CyberInfrastructure workshop CSG May Ann Arbor, Michigan.
DOE PI Meeting at BNL 1 Lightweight High-performance I/O for Data-intensive Computing Jun Wang Computer Architecture and Storage System Laboratory (CASS)
Large Scale Parallel File System and Cluster Management ICT, CAS.
A High-Performance Campus-Scale Cyberinfrastructure For Effectively Bridging End-User Laboratories to Data-Intensive Sources Presentation by Larry Smarr.
SAN DIEGO SUPERCOMPUTER CENTER SDSC's Data Oasis Balanced performance and cost-effective Lustre file systems. Lustre User Group 2013 (LUG13) Rick Wagner.
Slide 1 Defining and Understanding Campus Policies associated with Integrating a Science DMZ into the Campus Environment Moderator: Wendy Huntoon, KINBER.
February 27, 2007 University Information Technology Services Research Computing Craig A. Stewart Associate Vice President, Research Computing Chief Operating.
UW Madison Campus Network Security Strategy Campus Firewall Service Rick Keir DoIT Network Services
Running BLAST on the cluster system over the Pacific Rim.
Brunel University, School of Engineering and Design, Uxbridge, UB8 3PH, UK Henry Nebrensky (not a systems manager) SIRE Group.
“The Pacific Research Platform: a Science-Driven Big-Data Freeway System.” Big Data for Information and Communications Technologies Panel Presentation.
Enabling the Digital Campus
Advanced Research Computing Projects & Services at U-M
Cyberinfrastructure: Many Things to Many People Russ Hobby Program Manager Internet2.
PRP End-to-End Technical Requirements From Science Applications Frank Würthwein moderator 16 October 2015.
Accelerating High Performance Cluster Computing Through the Reduction of File System Latency David Fellinger Chief Scientist, DDN Storage ©2015 Dartadirect.
Data Infrastructure foundation of the European Cloud Initiative
SCIENCE_DMZ NETWORKS STEVE PERRY, DIRECTOR OF NETWORKS UNM PIYASAT NILKAEW, DIRECTOR OF NETWORKS NMSU.
Southern California Infrastructure Philip Papadopoulos Greg Hidley.
“Pacific Research Platform Science Drivers” Opening Remarks PRP Science Driver PI Workshop UC Davis March 23, 2016 Dr. Larry Smarr Director, California.
Data Infrastructure in the TeraGrid Chris Jordan Campus Champions Presentation May 6, 2009.
Preview: emerging UCLA IT areas of emphasis Enhance community among students Exploit IT for learning strengths Enlarge access to content rich resources.
Introduction to Data Analysis with R on HPC Texas Advanced Computing Center Feb
Accessing the VI-SEEM infrastructure
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING
Private Public FG Network NID: Network Impairment Device
What is HPC? High Performance Computing (HPC)
Campus Cyberinfrastructure
Geoffrey Fox, Shantenu Jha, Dan Katz, Judy Qiu, Jon Weissman
Status and Challenges: January 2017
Introduction to XSEDE Resources HPC Workshop 08/21/2017
Topics in Storage Research
NSF : CIF21 DIBBs: Middleware and High Performance Analytics Libraries for Scalable Data Science PI: Geoffrey C. Fox Software: MIDAS HPC-ABDS.
Versatile HPC: Comet Virtual Clusters for the Long Tail of Science SC17 Denver Colorado Comet Virtualization Team: Trevor Cooper, Dmitry Mishin, Christopher.
SDM workshop Strawman report History and Progress and Goal.
NCRIS Research Data Services
Big Data, Simulations and HPC Convergence
Presentation transcript:

Trust & File Systems Rick Wagner HPC Systems Manager

IDI: Building the Digital University Rick Wagner Frank K. Wuerthwein Rob Knight “…integrates compute & storage resources at SDSC with instruments and labs across campus…” 2 Focuses Integrative Omics Seamless Data & Computing 3 Components High-performance, scalable, central storage Campus backbone and the Prism network as a Science DMZ Core and embedded IT staff in a tiered model

Why Will This Succeed? Tiered Staffing Model Project funds central storage engineers and lab admins Labs are not customers, they’re part of the team! storage engineer lab admin researcher Project Team Lab ALab B

On Campus c = ∞ Parallel File System Lab A Lab B Comet ScienceDMZ

Lab ALab B Comet Scalable Parallel File System

OSS MDS ost0/labA-ost0 ost0/labB-ost0 ost0/comet-ost0 ost1/labA-ost1 ost1/labB-ost1 ost1/comet-ost1 mdt0/labA-mdt0 mdt0/labB-mdt0 mdt0/comet-mdt0 Lab A Lab B Comet /oasis/projects/labA /oasis/projects/labB /oasis/comet/scratch /oasis/projects/labA /oasis/projects/labB NID0 NID1 ZFS is magical! zpool dataset

Lack of Trust

Lustre nodemap Steve Simms, IU LUG2015 +shared keys!

Lab A Lab B Comet Scalable Parallel File System Lab C virtual cluster FIONA

Lab A Lab B Comet Scalable Parallel File System Lab C virtual cluster Lab D c != ∞ ∴ streaming IO FIONA

Feedback: Privacy Policy Feedback
Do Not Sell My Personal Information
About project: SlidePlayer Terms of Service

© 2025 SlidePlayer.com. Inc. All rights reserved.