Current Developer Resources at NCSA Don Petravick LSST All Hands Meeting August 16, 2017

Current Resources LSST-DEV General Purpose Login/Compute Node. Supports developers and staff. Primary Linux Machine for this use. https://developer.lsst.io/services/lsst-dev.html

LSST-DB Unmanaged Database Resource for Developers However it is backed up. Old machine, refresh under discussion https://developer.lsst.io/services/lsst-db.html

Verification Cluster Cluster of servers for LSST development work. Currently SLURM to support control_pool based HSC reprocessing. Over time, plan calls for transition to HT Condor. The Cluster is a “pile of batch nodes.” Resource will be shared with production, integration and DAC. https://developer.lsst.io/services/verification.html

/Datasets Datasets needed to support development. The original intention is invariant and long-lived datasets Is currently also used by HSC production. In 2018 Production file systems will be established https://developer.lsst.io/services/datasets.html

Site File Systems for Development https://developer.lsst.io/services/data_protection.html Home: login areas -- 30 day restore + DR Software: Project-shared software installations -- 30 day + DR SUI: Support the PDAC -- no protection Scratch: Ephemeral storage -- no protection, purged Project: Projects -- not purged, no protection Note: Object stores are not in the baseline for the LSST Data Facility.

Bulk Transfer for developers https://developer.lsst.io/services/ncsa_bulk_transfer.html Globus Online iRODS bbftp

LSST-WEB Database Schema Browser Doxygen Home directory web sharing Image retrieval Service pipeQA Sim Software meta-data Stripe 82 database account sign up.

Nebula https://developer.lsst.io/services/nebula/index.html Virtual machine environment for prototyping and exploration Code portability. Result of replan: containerization, not virtualization is the technical method for virtual environments for production. Kubernetes is the containerized application management platform.

ORCA https://developer.lsst.io/services/orchestration/index.html The orca framework remains available. Conforms to the “high throughput” model that is seen as the model for production services.

Other resources AA – LDAP, CI-logon, and OAuth LSST-DBDEV PDAC L1 Complete test stand Elastic and Container Testing. Data Backbone Testing Kubernetes testing Workflow and processing testing.

Support infrastructure Provisioning Monitoring Backups Documents Security

Tickets, please. Incident response and tickets: Service Manager Dashboard https://jira.lsstcorp.org/secure/Dashboard.jspa?selectPageId=12809 NCSA Tickets: https://jira.lsstcorp.org/issues/?filter=15322 NCSA Open Tickets: https://jira.lsstcorp.org/issues/?filter=15327 NCSA Unassigned Tickets: https://jira.lsstcorp.org/issues/?filter=15328

Special Resources Lsst-dbdev – Qserv Testing PDAC – SUI & QSERV Prototyping L1 complete test stand – L1 service, OCS, CDS, Camera DAQ, EFD Elastic and Container Testing Jupyter notebooks Firefly demo Data backbone Testing Workflow and processing testing.

Future Starting in 2017 we started to implement the production hardware services, in addition to maintaining the development and integration resourses. Reasonable efforts are made to keep development resources distinct from the production resources. There’s a talk tomorrow that will focus on the production resources.

Summary A number of types of development resources is in place. We see further development being in specific subproject support. Some resources are LSST funded contributions to a commons at NCSA. The most visible element is the Openstack. All resources need annual scrutiny for capacity, availability and refresh. Operational/Service-Oriented procedures are maturing, and work is underway to integrate with LSST ticket systems, etc.