R3 Kickoff Meeting Ocean Observatories Initiative Common Execution Infrastructure (CEI) Subsystem OOI CI System Architecture Team: 1.

Slides:



Advertisements
Similar presentations
AMUSE Autonomic Management of Ubiquitous Systems for e-Health Prof. J. Sventek University of Glasgow In collaboration.
Advertisements

ITIL: Service Transition
An Approach to Secure Cloud Computing Architectures By Y. Serge Joseph FAU security Group February 24th, 2011.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Infrastructure Kate Keahey, Tim Freeman, Alex Clemesha, David LaBissoniere,
A Java Architecture for the Internet of Things Noel Poore, Architect Pete St. Pierre, Product Manager Java Platform Group, Internet of Things September.
BETA!BETA! Building a secure private cloud on Microsoft technologies Private cloud security concerns Security & compliance in a Microsoft private cloud.
Agent-Based Acceptability-Oriented Computing International Symposium on Software Reliability Engineering Fast Abstract by Shana Hyvat.
EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space User Oriented Provisioning of Secure Virtualized.
A T AXONOMY AND S URVEY OF C LOUD C OMPUTING S YSTEMS Reporter: Steven Chen Date: 2010/10/27 1.
INTRODUCTION TO CLOUD COMPUTING CS 595 LECTURE 4.
Architecture overview 6/03/12 F. Desprez - ISC Cloud Context : Development of a toolbox for deploying application services providers with a hierarchical.
1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu
Microsoft ® Application Virtualization 4.5 Infrastructure Planning and Design Series.
Cloud Computing Systems Lin Gu Hong Kong University of Science and Technology Sept. 21, 2011 Windows Azure—Overview.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Microsoft ® Application Virtualization 4.6 Infrastructure Planning and Design Published: September 2008 Updated: February 2010.
CLOUD COMPUTING. IAAS / PAAS / SAAS LAYERS. Olena Matokhina Development and Consulting Team Lead 2 ABOUT PRESENTER.
VAP What is a Virtual Application ? A virtual application is an application that has been optimized to run on virtual infrastructure. The application software.
Towards auto-scaling in Atmosphere cloud platform Tomasz Bartyński 1, Marek Kasztelnik 1, Bartosz Wilk 1, Marian Bubak 1,2 AGH University of Science and.
Ocean Observatories Initiative Common Execution Environment Kate Keahey OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego,
Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over the Internet. Cloud is the metaphor for.
 Cloud computing  Workflow  Workflow lifecycle  Workflow design  Workflow tools : xcp, eucalyptus, open nebula.
OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep Ocean Observatories Initiative OOI CI Release 2 Life Cycle Objectives Review CyberPoPs & Network.
Cloud Computing 1. Outline  Introduction  Evolution  Cloud architecture  Map reduce operation  Platform 2.
Ocean Observatories Initiative Sensing and Acquisition (SA) Subsystem Overview Michael Meisinger September 29, 2009.
Ocean Observatories Initiative Common Execution Infrastructure (CEI) Overview Michael Meisinger September 29, 2009.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Environment Kate Keahey, Tim Freeman, Alex Clemesha, John Bresnahan, David.
OOI CI R2 Life Cycle Objectives Review Aug 30 - Sep Ocean Observatories Initiative OOI CI Release 2 Life Cycle Objectives Review Common Execution.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Common Execution Environment Kate Keahey, Tim Freeman, Alex Clemesha, John Bresnahan, David.
Large Scale Sky Computing Applications with Nimbus Pierre Riteau Université de Rennes 1, IRISA INRIA Rennes – Bretagne Atlantique Rennes, France
Ocean Observatories Initiative OOI Cyberinfrastructure Life Cycle Objectives Review January 8-9, 2013 R3 Life Cycle Objective Review for Common Execution.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger Life Cycle Architecture Review.
Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger September 29, 2009.
ANF Detection Network Dave Foster Guy de Wardener
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger Life Cycle.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Assessment John Graybeal, Michael Meisinger Life Cycle.
 Apache Airavata Architecture Overview Shameera Rathnayaka Graduate Assistant Science Gateways Group Indiana University 07/27/2015.
Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott VMware Presentation March 5, 2010.
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License Cloud Hosting Practices Lessons DuraSpace has learned Bill Branan Open Repositories.
The Aeolus Project Clouds flying in assembly David Lutterkort Red Hat, Inc.
System/SDWG Update Management Council Face-to-Face Flagstaff, AZ August 22-23, 2011 Sean Hardman.
Aneka Cloud ApplicationPlatform. Introduction Aneka consists of a scalable cloud middleware that can be deployed on top of heterogeneous computing resources.
Web Technologies Lecture 13 Introduction to cloud computing.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Sensing and Acquisition Subsystem Arjuna Balasuriya Life Cycle Architecture Review La Jolla,
Ocean Observatories Initiative Serving Ocean Model Data on the Cloud M. Meisinger, C. Farcas, E. Farcas, C. Alexander, M. Arrott, J. de La Beaujardière,
Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott VMware Presentation March 5, 2010.
Ocean Observatories Initiative OOI Cyberinfrastructure Common Execution Infrastructure Michael Meisinger OOI Cyberinfrastructure Life Cycle Objectives.
Building Cloud Solutions Presenter Name Position or role Microsoft Azure.
Ocean Observatories Initiative Common Operating Infrastructure (COI) Overview Michael Meisinger, Munindar Singh September 29, 2009.
© 2012 Eucalyptus Systems, Inc. Cloud Computing Introduction Eucalyptus Education Services 2.
Information Initiative Center, Hokkaido University North 11, West 5, Sapporo , Japan Tel, Fax: General.
Nimbus Update March 2010 OSG All Hands Meeting Kate Keahey Nimbus Project University of Chicago Argonne National Laboratory.
Structured Container Delivery Oscar Renalias Accenture Container Lead (NOTE: PASTE IN PORTRAIT AND SEND BEHIND FOREGROUND GRAPHIC FOR CROP)
Agenda  What is Cloud Computing?  Milestone of Cloud Computing  Common Attributes of Cloud Computing  Cloud Service Layers  Cloud Implementation.
ITIL: Service Transition
Infrastructure Orchestration to Optimize Testing
POW MND section.
StratusLab Final Periodic Review
StratusLab Final Periodic Review
“The HEP Cloud Facility: elastic computing for High Energy Physics” Outline Computing Facilities for HEP need to evolve to address the new needs of the.
AWS DevOps Engineer - Professional dumps.html Exam Code Exam Name.
Pass AWS-Certified-Solutions-Architect-Professional Certification in First Attempt - AWS-Certified-Solutions-Architect-Professional Valid Questions Answers - Realexamdumps.com
Get Amazon AWS-DevOps-Engineer-Professional Exam Real Questions - Amazon AWS-DevOps-Engineer-Professional Dumps Realexamdumps.com
Leigh Grundhoefer Indiana University
Automating Security in the Cloud
Azure and Partner Solution Enable Pay-Per-Hour IBM WebSphere Dev/Test Environments in Cloud MINI-CASE STUDY “Working with customers to migrate to cloud.
Cloud Computing: Concepts
Erik Vollebekk Application Architect
Presentation transcript:

R3 Kickoff Meeting Ocean Observatories Initiative Common Execution Infrastructure (CEI) Subsystem OOI CI System Architecture Team: 1

R3 Kickoff Meeting CEI Developers 2 4/14/ CEI Developer John Bresnahan Argonne National Lab (part-time) CEI Developer Patrick Armstrong University of Chicago CEI Developer Pierre Riteau University of Chicago (part-time) CEI Senior Developer Pierre Riteau University of Chicago

R3 Kickoff Meeting Subsystem Purpose Allow OOI applications and system to –Provide Highly Available (HA) services –Scale to demand Enact OOI deployment policies in elastic environment Provide a deployment foundation for OOI CI 3

R3 Kickoff Meeting Core System Structure: Service Layers 4

R3 Kickoff Meeting CEI Scope Elastic Computing Services –Implement elastic computing services to provide on-demand scaling and high availability. Execution Engine Catalog & Repository Services –Working with operations and ITV to develop and refine tools to upload and sync the different deployable type representations adapted to each site. Process Management Services –Provide the management services for policy-based process execution within specified deployable types intended to support the data distribution services; as such the processes are sequential and require primarily a process to resource match. Process Catalog & Repository Services –The Process Catalog and Repository Services maintain process definitions as well as lists active processes. Integration with the National Computing Infrastructure –Provide the capability to deploy OOI processing on the Amazon cloud services as well as academic clouds 5

R3 Kickoff Meeting High Availability and Scaling High Availability –Towards an always-on service model –Failures in outsourced resources –Providing a pool of replenishable compute resources Autoscaling –Provide resources for peaks in demand –Ensure good utilization during “valleys” in demand –Flexible resource mix 4/14/20154/14/20154/14/2015 6

R3 Kickoff Meeting Resources for HA and Scaling 4/14/20154/14/20154/14/ EPU Management Monitor and regulate set properties based on system-specific and application-specific metrics –Cloud resources are available on-demand, but any particular resource may fail at any time –Applications/processes can absorb new resources –Applications/processes can tolerate failures EPU

R3 Kickoff Meeting Managing Resources 8

R3 Kickoff Meeting EE ioncore 1.3 EPU Management Elastic Processing Unit (EPU) Management 9 EE ioncore 1.2 context-agent ou-agent EE matlab 6.1 context-agent ou-agent Decision Engine context-agent ou-agent Provisioner IaaS create instance AMQP Other DTRS CB

R3 Kickoff Meeting Making the EPU HA ou-agent EPU Worker Bootstrap EPU Dedicated DE Provisioner/DTRS IaaS create instance AMQP Other cloudinit.d

R3 Kickoff Meeting Managing Processes

R3 Kickoff Meeting Creating a Process I 12 Process Definition Registry Process Dispatcher EE type A instance Process Instance Registry request to activate process X ee-agent Decision Engine lookup launch enter AMQP Other

R3 Kickoff Meeting Creating a Process II 13 Process Definition Registry Process Dispatcher Provisioner/DTRS IaaS EE type A instance EPU Management Process Instance Registry request to activate process X ee-agent Decision Engine lookup launch enter request instance create instance AMQP Other

R3 Kickoff Meeting CC instance Inside an Execution Engine 14 EE type A instance context-agent ee-agent ou-agent supervisord Matlab script C C M CMR CMK CMKO datastream subscriptionresult Process Dispatcher EPU Management Package Server process (adapter) 1 AMQP Other C – create M – monitor R – restart K – kill O – I/O

R3 Kickoff Meeting Adventures in Availability Time to repair (TTR) –Diagnosis –Time to scale (TTS) PENDING (request) STARTED (deployment) RUNNING (contextualization) 4/14/20154/14/20154/14/ A = MTBF MTBF+MTTR Mean time between failures Mean time to repair TTS: preliminary results for 2,000 VMs provisioned on AWS EC2

R3 Kickoff Meeting R3 Scope Process management –Activation and validation –New execution site registration Integration with National Infrastructure –Framework for integration of academic cloud providers, TeraGrid and OSG –Integration with Microsoft cloud 16

R3 Kickoff Meeting R3 Activities Refine/change scope to achieve a complete and maintainable system Decide on specific solutions for R3 scope 17

R3 Kickoff Meeting Questions? 18