OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Environment Kate Keahey, Tim Freeman, Alex Clemesha, John Bresnahan, David.

Slides:

Advertisements

Similar presentations

Current methods for negotiating firewalls for the Condor ® system Bruce Beckles (University of Cambridge Computing Service) Se-Chang Son (University of.

Advertisements

Categories of I/O Devices

NSF ANNUAL REVIEW June 2010 Ocean Observatories Initiative August Release 1 Life Cycle Architecture (LCA) Review Status of CI Project Matthew Arrott.

R3 Kickoff Meeting Ocean Observatories Initiative Common Execution Infrastructure (CEI) Subsystem OOI CI System Architecture Team: 1.

CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.

WHAT IS AN OPERATING SYSTEM? An interface between users and hardware - an environment "architecture ” Allows convenient usage; hides the tedious stuff.

An Approach to Secure Cloud Computing Architectures By Y. Serge Joseph FAU security Group February 24th, 2011.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Infrastructure Kate Keahey, Tim Freeman, Alex Clemesha, David LaBissoniere,

Copyright 2009 FUJITSU TECHNOLOGY SOLUTIONS PRIMERGY Servers and Windows Server® 2008 R2 Benefit from an efficient, high performance and flexible platform.

OPERATING SYSTEM OVERVIEW

MCTS Guide to Microsoft Windows Server 2008 Network Infrastructure Configuration Chapter 8 Introduction to Printers in a Windows Server 2008 Network.

Architecture overview 6/03/12 F. Desprez - ISC Cloud Context : Development of a toolbox for deploying application services providers with a hierarchical.

Cloud Computing Systems Lin Gu Hong Kong University of Science and Technology Sept. 21, 2011 Windows Azure—Overview.

Microsoft ® Application Virtualization 4.6 Infrastructure Planning and Design Published: September 2008 Updated: February 2010.

A Brief Overview by Aditya Dutt March 18 th ’ Aditya Inc.

Ocean Observatories Initiative Common Execution Environment Kate Keahey OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego,

OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep Ocean Observatories Initiative OOI CI Release 2 Life Cycle Objectives Review CyberPoPs & Network.

 What is an operating system? What is an operating system?  Where does the OS fit in? Where does the OS fit in?  Services provided by an OS Services.

Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009.

Presented by Xiaoyu Qin Virtualized Access Control & Firewall Virtualization.

OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep Ocean Observatories Initiative OOI CI Release 2 Life Cycle Objectives Review Common Execution.

EPE Release 2 IOC Review August 7, 2012 Ocean Observatories Initiative OOI EPE Release 2 Initial Operating Capability Review System Development Overview.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Environment Kate Keahey, Tim Freeman, Alex Clemesha, John Bresnahan, David.

Large Scale Sky Computing Applications with Nimbus Pierre Riteau Université de Rennes 1, IRISA INRIA Rennes – Bretagne Atlantique Rennes, France

Challenges towards Elastic Power Management in Internet Data Center.

608D CloudStack 3.0 Omer Palo Readiness Specialist, WW Tech Support Readiness May 8, 2012.

Ocean Observatories Initiative OOI Cyberinfrastructure Life Cycle Objectives Review January 8-9, 2013 R3 Life Cycle Objective Review for Common Execution.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger Life Cycle Architecture Review.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Construction Plan John Graybeal Life Cycle Architecture Review La Jolla, CA.

OOI CI EOI LCA REVIEW December 13, 2010 Ocean Observatories Initiative External Observatory Integration Christopher Mueller Life Cycle Architecture Review.

Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger September 29, 2009.

Some Design Notes Iteration - 2 Method - 1 Extractor main program Runs from an external VM Listens for RabbitMQ messages Starts a light database engine.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative External Observatory Integration Christopher Mueller, Matt Arrott, John Graybeal Life Cycle.

FailSafe SGI’s High Availability Solution Mayank Vasa MTS, Linux FailSafe Gatekeeper

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger Life Cycle.

Module 7: Managing Message Transport. Overview Introduction to Message Transport Implementing Message Transport.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Assessment John Graybeal, Michael Meisinger Life Cycle.

 Apache Airavata Architecture Overview Shameera Rathnayaka Graduate Assistant Science Gateways Group Indiana University 07/27/2015.

Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott VMware Presentation March 5, 2010.

What is SAM-Grid? Job Handling Data Handling Monitoring and Information.

Visual Studio Windows Azure Portal Rest APIs / PS Cmdlets US-North Central Region FC TOR PDU Servers TOR PDU Servers TOR PDU Servers TOR PDU.

1: Operating Systems Overview 1 Jerry Breecher Fall, 2004 CLARK UNIVERSITY CS215 OPERATING SYSTEMS OVERVIEW.

Ocean Observatories Initiative CEI Demonstrations Tim Freeman OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego, CA February.

Grid Computing Environment Shell By Mehmet Nacar Las Vegas, June 2003.

Smart Home Technologies

Data Communications and Networks Chapter 9 – Distributed Systems ICT-BVF8.1- Data Communications and Network Trainer: Dr. Abbes Sebihi.

Aneka Cloud ApplicationPlatform. Introduction Aneka consists of a scalable cloud middleware that can be deployed on top of heterogeneous computing resources.

Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger, Munindar Singh, Von Welch OOI Cyberinfrastructure.

Microsoft ® Forefront ™ Identity Manager 2010 Infrastructure Planning and Design Published: June 2010.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Sensing and Acquisition Subsystem Arjuna Balasuriya Life Cycle Architecture Review La Jolla,

Ocean Observatories Initiative Serving Ocean Model Data on the Cloud M. Meisinger, C. Farcas, E. Farcas, C. Alexander, M. Arrott, J. de La Beaujardière,

Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott VMware Presentation March 5, 2010.

Ocean Observatories Initiative OOI Cyberinfrastructure Life Cycle Objectives Review January 8-9, 2013 Scientific Workflows for OOI Ilkay Altintas Charles.

Ocean Observatories Initiative OOI Cyberinfrastructure Common Execution Infrastructure Michael Meisinger OOI Cyberinfrastructure Life Cycle Objectives.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative External Observatory Integration: Christopher Mueller Life Cycle Objectives Review La Jolla,

Ocean Observatories Initiative Common Operating Infrastructure (COI) Overview Michael Meisinger, Munindar Singh September 29, 2009.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative User Experience Susanne Jul, Carolanne Fisher, Tom Auer, Chris Quintana Life Cycle Architecture.

Ocean Observatories Initiative Integrating Marine Observatories into a System-of-Systems: Messaging in the US Ocean Observatories Initiative M. Arrott,

Cloud Computing from a Developer’s Perspective Shlomo Swidler CTO & Founder mydrifts.com 25 January 2009.

Chapter 4: server services. The Complete Guide to Linux System Administration2 Objectives Configure network interfaces using command- line and graphical.

© 2012 Eucalyptus Systems, Inc. Cloud Computing Introduction Eucalyptus Education Services 2.

Nimbus Update March 2010 OSG All Hands Meeting Kate Keahey Nimbus Project University of Chicago Argonne National Laboratory.

AMSA TO 4 Advanced Technology for Sensor Clouds 09 May 2012 Anabas Inc. Indiana University.

CSCE 548 Secure Software Development Risk-Based Security Testing

Building a Database on S3

Simplified Development Toolkit

OOI Cyberinfrastructure Integrated Elaboration Demo

Presentation transcript:

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Environment Kate Keahey, Tim Freeman, Alex Clemesha, John Bresnahan, David LaBissoniere, Life Cycle Architecture Review La Jolla, CA

OOI CI LCA REVIEW August 2010 Subsystem Purpose Provide highly available services Rapidly provision resources Scale to demand

OOI CI LCA REVIEW August 2010 R1 Use Cases UC.R1.16: Scale the processing A load is put on the system Additional demand is recognized via different sensors Message queue length, CPU loads, disk usage System scales up to meet increased demand System scales down when demand goes away UC.R1.25: Assure reliability Failures happen Remedial actions happen No significant impact on observatory operation

OOI CI LCA REVIEW August 2010 Architecture Overview

OOI CI LCA REVIEW August 2010 More Architecture Overview

OOI CI LCA REVIEW August 2010 Even More Architecture Overview (HA Provisioning)

OOI CI LCA REVIEW August 2010 Even More Bootstrapping

OOI CI LCA REVIEW August 2010 Implementation/Achievement Status All major components implemented: Provisioner, Controller, Planner, Sensor Aggregator Integrated with ION Tested on infrastructure ranging from Magellan to EC2 Needing refinement: exchange points, EPU adapters, cc-agent, use of Registry, Planner, Image building and management Need to put together HA-Provisioner and bootstrapping Draft user and administrator process

OOI CI LCA REVIEW August 2010 Technology Choices Services: Python, Twisted Messaging: ION container (AMQP) Infrastructure: Nimbus IaaS, EC2 IaaS, libcloud, Nimboss, boto Contextualiztion: Nimbus Context Broker/agent, Nimboss, Chef Solo, Fabric Special: txrabbitmq, Twotp (queue length sensor)

OOI CI LCA REVIEW August 2010 Use Cases Addressed and Demonstrated UC.R1.16: Scale the processing UC.R1.25: Assure reliability

OOI CI LCA REVIEW August 2010 The Testfest Tests run on EC2: CEI infrastructure deployed on two small instances: Provisioner, DTRS, EPU_Controller, Sensor Aggregator, client RabbitMQ server on a large instance with elastic IP UC Context Broker instance Fully experimental (except for queue length) Target: scale the scenarios to up to 1000 VMs

OOI CI LCA REVIEW August 2010 Scale the Processing Test1 70 jobs Submitted over 28 minutes, 5 jobs every 2 minutes Long job duration, one job per VM Test 2 70 jobs Submitted at the same time Long job duration, one job per VM

OOI CI LCA REVIEW August 2010 Assure Reliability How does the system react to failure? Killing 5 VMs in 5 minute intervals Saturating the system with 10 second jobs Bounded policy: 40 VMs

OOI CI LCA REVIEW August 2010 Technology Challenges Most significant resolved issues: Semantic differences between IaaS providers Issues in contextualization recepies Multiple simple bugs Most significant unresolved issues: AMQP connections closed unexpectedly Currently prevents us from running at scale RabbitMQ crashes when used by multiple clients Inspecting queue remotely issue Non-concurrent container issue Pulling work issue

OOI CI LCA REVIEW August 2010 Plan for Construction Complete feature set Continue evaluation and improvement Make code ready for users Refine documentation Continue integration with other subsystems Acceptance tests and harness

OOI CI LCA REVIEW August 2010 Significant Risks