1. Bridges and Clouds
Sergiu Sanielevici, PSC Director of User Support for Scientific Applications
October 12, 2017
© 2017 Pittsburgh Supercomputing Center

2. Support for cloud-like services on Bridges
HPCaaS, BDaaS, and (coming) AIaaS, particularly through gateways
Support for OSG-style workflows (Glidein, CMVFS)
Containers
Singularity for most applications
Docker for certain, vetted software stacks
Very flexible software environment supporting, e.g., Anaconda, Spark, Hadoop, etc.

3. Planned “Social Weather Service (SWS)” U. Pittsburgh+U. Connecticut
High Performance Data Analytics (HPDA) infrastructure: “Our infrastructure will leverage PSC’s Bridges resource for high-performance data analytics, data linkage, reliability assessment, and data fusion. Bridges is a uniquely capable HPC system designed to enable new kinds of research by providing a flexible software environment, interactivity, and nodes with up to 12TB RAM to allow seamless scaling of in-memory database operations and data analytics.”
Cloud infrastructure: “For data assimilation (e.g., geotagged Twitter data) and elastically scaling routine user queries, we will integrate AWS into the SWS framework. That will include interfaces from both our local on- premises cluster and from Bridges, together with storing mirrors of the Data Repositories for cloud-local analytics that requires less memory.”

4. Databases and Distributed/Web Services
Dedicated database nodes power persistent relational and NoSQL databases
Support data management and data-driven workflows
SSDs for high IOPs; HDDs for high capacity
Dedicated web server nodes
Enable distributed, service-oriented architectures
High-bandwidth connections to XSEDE and the Internet
(examples)

5. High-Productivity Programming

6. Spark, Hadoop & Related Approaches
Bridges’ large memory opens new possibilities for Spark
Bridges enables workflows that integrate Spark/Hadoop, HPC, GPU, and/or large shared-memory components.

7. Deep Learning Frameworks on Bridges
GPU

8. Virtualization and Containers
Bridges leverages OpenStack to provision resources, between bare-metal, Spark/Hadoop, and VM node configurations.
Virtual Machines (VMs) enable flexibility, security, customization, reproducibility, ease of use, and interoperability with other services.
User demand to date has been for:
persistent database and web server installations to develop data-intensive, distributed applications
containers for custom software stacks and portability
Hadoop and Spark mostly for education and training

9. OpenStack Virtualization @ PSC
PSC Managed
Non-Root user accounts for project
User-support staff works with researchers/handles deploy
Customize software stack, filesystem requirements, etc.
Dual Home network access
VMs can talk to BRIDGES
shared filesystems (Lustre/Slash2)
compute nodes
Issue jobs to SLURM (Community Gateways)
Deployment setup automated w/Puppet & Ansible
Uses cloud provisioned SSH key to automate VM setup
Packages, OPA configuration, Kerberos user accounts, etc.
QCOW2 files are simple and can be backed up!

10. OpenStack Virtualization @ PSC
User Managed
Full root access
Complex Software stacks require time
Limited to special use cases/users
Requests/reasoning are vetted.
No shared file systems w/BRIDGES
Need Ticket based auth on FS
Wild west, you’re on your own!
Assume general knowledge of Linux administration practices
Security Implications
Scanning VMs
Root SSH Key for detailed profiling
User Agreements / Signatures
Talk about some User Managed VMs becoming PSC Managed

11. OpenStack Virtualization @ PSC
Recent Additions:
Cinder Capacity Storage
40TB JBOD
Block Storage for VMs
Cinder Performance Storage
2x960GB SSD Nodes
Better Puppet Integration w/Bridges
Rebuild VMs from Puppet or Snapshots
Future Goals:
Upgrade -> Ocata!
High Availability Support
Better OPA Implementation
Automated Snapshots/Backups!
Storage added for User Managed VMs