Deploying Experiments with Raven Scott Baker SB-Software John H. Hartman University of Arizona

Raven Components Raven – Experiment administration Owl – Experiment monitoring Repository – Holds metadata / packages / DBs Tempest – Remote deployment daemon Stork – Package management – Like yum – Enhanced security, hierarchical trust

Raven Overview

Experiment Flow

Raven Experiment Container Subdirectory structure – Packages/ rpms, tarballs, etc – Users/ other people you trust – Templates/ templates for config files, etc – Tpfiles/trust specification (auto-gen) – Tempest/ tempest configuration (auto-gen) – Config/per-slice config files (auto-gen)

Raven Advantages Secure – “tpfiles” contain hashes of trusted packages – Tpfiles, config files, etc., signed by experimenter Experiments can span slices and partners – Example: PlanetLab, GpENI demo Single-command experiment deployment Supports Cluster-B GENI credentials

Deploying an Experiment Create a new experiment container – raven create – (Raven will ask you for experiment name, keys, etc) Copy in your package(s) – cp /home/mypackage.tar.gz packages/ Publish it – raven publish

Raven Publish Steps

owld client scripts apache/owl mysql browser apache/owl server display HTTP Owl

Status Report Raven tool implemented and deployed Owl tool implemented and deployed Google maps interface to Owl Migrated repository to mod_python Modified SFA authentication to explicitly sign arguments iftd transport service – work continuing

Live Demo cd ~ mkdir my_experiment cd my_experiment raven create –import ~/gec6demo.conf ls ~/make_helloworld.sh raven publish

Live Demo (1) $ cd ~ $ mkdir my_experiment

Live Demo (2) $ raven create Experiment name: [None] ? my_experiment Location of your private key: [None] ? ~/.sfi/bakers.pkey A GENI Credential file may be used to automatically upload files to the Raven repository. This file is optional, but without it you will be responsible for manually uploading the files. Location of GENI cred file: [None] ? ~/.sfi/bakers.cred Raven may be configured to manage the config files on your slices for you. You may enter multiple slice names separated by commas. Entering no slice names will cause packages and tpfiles to be uploaded, but not slice configuration files. Slices that should be managed by this experiment: [] ? arizona_client2

Live Demo (3) The packages.pacman file controls which packages will be installed on your nodes. This tool can be configured to automatically manage this file, by installing all of your packages on all of your nodes. Automatically manage your packages.pacman (y/n): [y] ? Automatically install and upgrade stork (y/n): [y] ? Automatically install and upgrade the owl slice management service (y/n): [y] ? By default, raven will attempt to install/upgrade all packages that you place in the packages directory. You can choose to not automatically install some packages, in which case the packages will be made available for dependency resolution, but not automatic installation. Package names to not install: [] ?

Live Demo (4) $ mkdir helloworld $ echo "date >> /tmp/helloworld.txt" > helloworld/autorun.sh $ tar -czf helloworld-1.0.tar.gz helloworld $ cp helloworld-1.0.tar.gz ~/my_experiment/packages/

Live Demo (5) raven publish – trusting: helloworld-1.0.tar.gz – trusting user: stork – trusting user: fedora8 – adding to packages.pacman: helloworld – copying: helloworld-1.0.tar.gz – signing: my_experiment.tpfile – signing: my_experiment.packages.pacman – signing: arizona_client2.stork.conf – uploading: my_experiment.fda12c53d8a fae38fcb0671efb packages.pac man True – uploading: helloworld-1.0.tar.gz True uploading: arizona_client2.fda12c53d8a fae38fcb0671efb stork.conf True uploading: my_experiment.fda12c53d8a fae38fcb0671efb tpfile True

Owl Monitoring Framework Extensible – scripts.d – directory of monitoring scripts Client/server – Database on raven repository Currently working on security and visualization