CernVM WebAPI CernVM Users Workshop 2015 Ioannis Charalampidis, PH/SFT CERN, 5 March 2015.

Slides:



Advertisements
Similar presentations
FI-WARE Testbed Access Control temporary solution.
Advertisements

Kyle Thurow, Kyle Neuschaefer, Alexander Matusiak, and Justin Carroll.
© 2012 IBM Corporation Architecture of Quantum Folsom Release Yong Sheng Gong ( 龚永生 ) gongysh #openstack-dev Quantum Core developer.
©2009 Justin C. Klein Keane PHP Code Auditing Session 3 – Tools of the Trade & Crafting Malicious Input Justin C. Klein Keane
Apache Web Server Quick and Dirty Steve Gibbard for SANOG 16 (Originally by Joel Jaeggli for AfNOG 2007) ‏
IT:Network:Applications VIRTUAL DESKTOP INFRASTRUCTURE.
Systems Analysis and Design in a Changing World, 6th Edition 1 Chapter 6.
1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu
MODULE 2: INSTALLING UNIDESK. Agenda Understanding Unidesk components Basic Installation of Unidesk Licensing.
Worker Role Web Role Web Role VM Role Control Abstraction (i.e. Less IT & Less Plumbing Code) Admin Web / Worker Role VM Role Web / Worker Role.
Presented by…. Group 2 1. Programming language 2Introduction.
System Center 2012 Setup The components of system center App Controller Data Protection Manager Operations Manager Orchestrator Service.
Web Applications Harry R. Erwin, PhD University of Sunderland CIT304/CSE301.
Building service testbeds on FIRE D5.2.5 Virtual Cluster on Federated Cloud Demonstration Kit August 2012 Version 1.0 Copyright © 2012 CESGA. All rights.
Testing Tools using Visual Studio Randy Pagels Sr. Developer Technology Specialist Microsoft Corporation.
Screen Snapshot Service Kurt Biery SiTracker Monitoring Meeting, 23-Jan-2007.
1 port BOSS on Wenjing Wu (IHEP-CC)
Model a Container Runtime environment on Your Mac with VMware AppCatalyst VMworld Fabio Rapposelli
PHP With Oracle 11g XE By Shyam Gurram Eastern Illinois University.
Presented by: Sanketh Beerabbi University of Central Florida COP Cloud Computing.
| nectar.org.au NECTAR TRAINING Module 5 The Research Cloud Lifecycle.
608D CloudStack 3.0 Omer Palo Readiness Specialist, WW Tech Support Readiness May 8, 2012.
ETICS All Hands meeting Bologna, October 23-25, 2006 NMI and Condor: Status + Future Plans Andy PAVLO Peter COUVARES Becky GIETZEL.
Apache Web Server Quick and Dirty for AfNOG 2015 (Originally by Joel Jaeggli for AfNOG 2007) ‏
Copyright © cs-tutorial.com. Overview Introduction Architecture Implementation Evaluation.
Apache Web Server Quick and Dirty Evelyn NAMARA for AfNOG 2014 (Originally by Joel Jaeggli for AfNOG 2007) ‏
1 Taverna CISTIB Ernesto Coto Taverna Open Workshop, October 2014.
Apache Web Server Quick and Dirty Ayitey Bulley for AfNOG 2011 (Originally by Joel Jaeggli for AfNOG 2007) ‏
Apache Web Server Quick and Dirty Kevin G. Chege for AfNOG 2013 (Originally by Joel Jaeggli for AfNOG 2007) ‏
HTML5 AND THE FUTURE JAVASCRIPT PLATFORM Marcelo Lopez Ruiz Senior Software Design Engineer Microsoft Corporation.
NA61/NA49 virtualisation: status and plans Dag Toppe Larsen CERN
Federating PL-Grid Computational Resources with the Atmosphere Cloud Platform Piotr Nowakowski, Marek Kasztelnik, Tomasz Bartyński, Tomasz Gubała, Daniel.
| nectar.org.au NECTAR TRAINING Module 5 The Research Cloud Lifecycle.
Cloud-access Author: Riccardo Bruno. cloud-access flow web portal A user accesses through any device to a portal requesting access to an interactive application.
2012 Objectives for CernVM. PH/SFT Technical Group Meeting CernVM/Subprojects The R&D phase of the project has finished and we continue to work as part.
1 Volunteer Computing at CERN past, present and future Ben Segal / CERN (describing the work of many people at CERN and elsewhere ) White Area lecture.
The overview How the open market works. Players and Bodies  The main players are –The component supplier  Document  Binary –The authorized supplier.
Selenium server By, Kartikeya Rastogi Mayur Sapre Mosheca. R
SFT Summer Student Proposals 9 th February CERN Summer Student Programme  PH has been allocated 91 MS students (SFT quota 6-7)  submit request.
Predrag Buncic (CERN/PH-SFT) Software Packaging: Can Virtualization help?
Virtual Machines Module 2. Objectives Define virtual machine Define common terminology Identify advantages and disadvantages Determine what software is.
Predrag Buncic (CERN/PH-SFT) CernVM Status. CERN, 24/10/ Virtualization R&D (WP9)  The aim of WP9 is to provide a complete, portable and easy.
Computing Facilities CERN IT Department CH-1211 Geneva 23 Switzerland t CF Cluman: Advanced Cluster Management for Large-scale Infrastructures.
RI EGI-TF 2010, Tutorial Managing an EGEE/EGI Virtual Organisation (VO) with EDGES bridged Desktop Resources Tutorial Robert Lovas, MTA SZTAKI.
Installation Guacamole Is a web application that provides access to desktop environments using remote desktop protocols (such as VNC or RDP); Installation.
CernVM and Volunteer Computing Ivan D Reid Brunel University London Laurence Field CERN.
If it’s not automated, it’s broken!
ArcGIS for Server Security: Advanced
ALICE & Clouds GDB Meeting 15/01/2013
Use of HLT farm and Clouds in ALICE
TrueSight Operations Management 11.0 Architecture
Volunteer Computing for Science Gateways
Users and Administrators
Dag Toppe Larsen UiB/CERN CERN,
Dag Toppe Larsen UiB/CERN CERN,
Docker Birthday #3.
ATLAS Cloud Operations
(Originally by Joel Jaeggli for AfNOG 2007)‏
R
Virtualization in the gLite Grid Middleware software process
CernVM Status Report Predrag Buncic (CERN/PH-SFT).
OpenStack Ani Bicaku 18/04/ © (SG)² Konsortium.
OPNFV Arno Installation & Validation Walk-Through
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Introduction to Apache
Ivan Reid (Brunel University London/CMS)
ARCHITECTURE OVERVIEW
Running C# in the browser
Users and Administrators
The Future of Database Development (with containers)
Presentation transcript:

CernVM WebAPI CernVM Users Workshop 2015 Ioannis Charalampidis, PH/SFT CERN, 5 March 2015

Agenda Origins of CernVM WebAPI Components of CernVM WebAPI Running multiple projects in the same VM Hands-on CernVM WebAPI Ioannis Charalampidis - CernVM WebAPI 2

CernVM WebAPI Origins 2011 : 2.0 – BOINC Project First time where virtualization is used 1 1 Ioannis Charalampidis - CernVM WebAPI 3

CernVM WebAPI Origins Conceived as an additional burden to some volunteers 1 1 Ioannis Charalampidis - CernVM WebAPI 4

CernVM WebAPI Origins Want to volunteer BOINC Users Mostly with good computer skills Can we also target the rest of the audience? 1 1 Ioannis Charalampidis - CernVM WebAPI 5

CernVM WebAPI Origins Web apps are simple, run everywhere and require no manual installations 1 1 Ioannis Charalampidis - CernVM WebAPI 6

CernVM WebAPI Origins Virtualization is widespread: Code it once, deploy it everywhere 1 1 Ioannis Charalampidis - CernVM WebAPI 7

CernVM WebAPI Origins 1 1 Ioannis Charalampidis - CernVM WebAPI 8

CernVM WebAPI Origins Raw computing power available to web apps 1 1 Ioannis Charalampidis - CernVM WebAPI 9

CernVM WebAPI Start and Control Virtual Computing Resources through your web browser Web Virtualization 1 1 Ioannis Charalampidis - CernVM WebAPI 10

CernVM WebAPI Does all the heavy-lifting for the user : Missing hypervisor? Install it! Improperly configured hypervisor? Fix it! Missing resources? Download them! One-click solution to desktop virtualization 1 1 Ioannis Charalampidis - CernVM WebAPI 11

CernVM WebAPI // Request API Access CVM.startCVMWebAPI(function(plugin) { // Open Session plugin.requestSession(" function(session) { // Start VM session.start(); }); Javascript code for starting a VM 1 1 Ioannis Charalampidis - CernVM WebAPI 12

Under the Hood Javascript Library Javascript Library libCernVM Library libCernVM Library WebAPI Daemon WebSocket CernVM WebAPI components Web BrowserSystem Daemon 2 2 Ioannis Charalampidis - CernVM WebAPI 13

Under the Hood - libCernVM libCernVM Library libCernVM Library Session { config } VM Instance libCernVM interfaces with the Hypervisor Ioannis Charalampidis - CernVM WebAPI 14

Under the Hood - libCernVM Missing Power Off Runnig Saved CreateStartSave DestroyStopResume Each session is implemented as an FSM 2 2 Ioannis Charalampidis - CernVM WebAPI 15

Under the Hood – Javascript Javascript Library Javascript Library High-level calls cvmwebapi.js provides the higher-level abstraction Install Logic Lower-level ws:// protocol Lower-level ws:// protocol 2 2 Ioannis Charalampidis - CernVM WebAPI 16

Under the Hood – Javascript Embedded UI First-time install instructions Micro-UI injected in the website for guiding the user 2 2 Ioannis Charalampidis - CernVM WebAPI 17

Under the Hood – Daemon WebAPI Daemon WebAPI Daemon WebServer (mongoose) WebServer (mongoose) Lower-level ws:// protocol Lower-level ws:// protocol Auth Logic Auth Logic libCernVM The daemon is just the glue between the javascript interface and libCernVM 2 2 Ioannis Charalampidis - CernVM WebAPI 18

Security & Trust Be secure, or you are a potential botnet node … 2 2 Ioannis Charalampidis - CernVM WebAPI 19

Security & Trust … Public Key domain.com Private Key domain.com WebAPI Public Key WebAPI Private Key Sign Validate VMCP Response Sign Validate WebAPI Daemon CERNdomain.com 2 2 Ioannis Charalampidis - CernVM WebAPI 20

Security & Trust - VMCP WebAPI Daemon WebAPI Daemon 1) Open Session Referrer: domain.com Web Server vmp.domain.co m Web Server vmp.domain.co m VMCP: vmcp.domain.com/vm2 2) Request Config 3) Sign response Private Key domain.com libCernVM 4) Forward request 5) Respond 2 2 Ioannis Charalampidis - CernVM WebAPI 21

Hypervisor Interaction with the VM Virtual Machine WebAPI Setup port forwarding Check Notify WebServer Browser Application Connect to API Port 2 2 Ioannis Charalampidis - CernVM WebAPI 22

WebAPI in Production CERN 60 – Public Computing Challenge 2 2 Ioannis Charalampidis - CernVM WebAPI 23

WebAPI in Production During the challenge detailed analytics were collected 2 2 Ioannis Charalampidis - CernVM WebAPI 24

WebAPI in Production 2 2 Used in cernvm-online.cern.ch For testing your contextualization image directly from the browser Used in Virtual Atom Smasher game For starting the worker nodes (simulation agents) that players need Ioannis Charalampidis - CernVM WebAPI 25

Multiple projects with WebAPI A VM can be easily booted from the browser However, starting a VM per project can consume resources Why not use lightweight virtualization (Linux Containers) inside a single VM? A ‘dumb’ scheduler: DumbQ Ioannis Charalampidis - CernVM WebAPI 26

Multiple projects with WebAPI # Server configuration project1:50:sft.cern.ch:sft.cern.ch/bootstraps/pj1/init.sh Project2:30:sft.cern.ch:sft.cern.ch/bootstraps/pj2/init.sh cernvm-fork pj1/init.sh cernvm-fork pj1/init.sh Free CPU Slot 50% Chance 30% Chance 3 3 Ioannis Charalampidis - CernVM WebAPI 27

Multiple projects with WebAPI Very simple integration with your current setup Your script runs in a standard CernVM environment This is not a scheduler, you have to use your own (as your current set-up perhaps) DumbQ provides the information required to identify the volunteer 3 3 Ioannis Charalampidis - CernVM WebAPI 28

Hands-on (?) Step-by-step tutorial: webapi/wiki/Tutorial-Intro 4 4 Ioannis Charalampidis - CernVM WebAPI 29