MESOS.

Slides:



Advertisements
Similar presentations
Paging: Design Issues. Readings r Silbershatz et al: ,
Advertisements

Begin $100 $200 $300 $400 $500 GraphsEconomicEquations Unit 1 Unit 3 KeyTerms Unit 2.
SLA-Oriented Resource Provisioning for Cloud Computing
Measuring Costs and Benefits Measuring Benefits and Costs (See Chap 4): –Consumers’ Willingness to Pay (WTP) –Consumer Surplus (CS) –Producers’ Surplus.
Session 2 Economic Systems Disclaimer: The views expressed are those of the presenters and do not necessarily reflect those of the Federal Reserve Bank.
Beyond Dominant Resource Fairness David Parkes (Harvard) Ariel Procaccia (CMU) Nisarg Shah (CMU)
Meeting Service Level Objectives of Pig Programs Zhuoyao Zhang, Ludmila Cherkasova, Abhishek Verma, Boon Thau Loo University of Pennsylvania Hewlett-Packard.
The "Almost Perfect" Paper -Christopher Francis Hyma Chilukiri.
Reciprocal Resource Fairness: Towards Cooperative Multiple-Resource Fair Sharing in IaaS Clouds School of Computer Engineering Nanyang Technological University,
Why static is bad! Hadoop Pregel MPI Shared cluster Today: static partitioningWant dynamic sharing.
Elasticity and Government Excise Tax Revenue Activity 21.
1 Managing Revenues in Regulated Industries Rate Design May 2008 Richard Soderman Director-Legislative Policy and Strategy.
Wrapping UP Insurance Let’s Review Moral Hazard With health insurance, the amount of expenditures may depend on whether you have insurance. Suppose that.
The Weighted Proportional Allocation Mechanism Milan Vojnović Microsoft Research Joint work with Thành Nguyen Harvard University, Nov 3, 2009.
Copyright © 2004 South-Western 7 Consumers, Producers, and the Efficiency of Markets.
CPS216: Advanced Database Systems (Data-intensive Computing Systems) How MapReduce Works (in Hadoop) Shivnath Babu.
Scheduling a Large DataCenter Cliff Stein Columbia University Google Research June, 2009 Monika Henzinger, Ana Radovanovic Google Research.
Scheduling of Flexible Resources in Professional Service Firms Arun Singh CS 537- Dr. G.S. Young Dept. of Computer Science Cal Poly Pomona.
1 CHAPTER 12 Factor Markets and the Distribution of Income PowerPoint® Slides by Can Erbil © 2004 Worth Publishers, all rights reserved.
A Dynamic MapReduce Scheduler for Heterogeneous Workloads Chao Tian, Haojie Zhou, Yongqiang He,Li Zha 簡報人:碩資工一甲 董耀文.
Incentivizing Sharing in Realtime D2D Streaming Networks: A Mean Field Game Perspective Jian Li Texas A&M University April 30 th, 2015 Jointly with R.
AN INTRODUCTION TO THE OPERATIONAL ANALYSIS OF QUEUING NETWORK MODELS Peter J. Denning, Jeffrey P. Buzen, The Operational Analysis of Queueing Network.
November , 2009SERVICE COMPUTATION 2009 Analysis of Energy Efficiency in Clouds H. AbdelSalamK. Maly R. MukkamalaM. Zubair Department.
Heterogeneity and Dynamicity of Clouds at Scale: Google Trace Analysis [1] 4/24/2014 Presented by: Rakesh Kumar [1 ]
Consumer Behavior Conjoint analysis Conjoint Analysis.
Pairwise Document Similarity in Large Collections with MapReduce Tamer Elsayed, Jimmy Lin, and Douglas W. Oard Association for Computational Linguistics,
Ch. 4 Memory Mangement Parkinson’s law: “Programs expand to fill the memory available to hold them.”
Dominant Resource Fairness: Fair Allocation of Multiple Resource Types Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, Ion.
Scheduling Periodic Real-Time Tasks with Heterogeneous Reward Requirements I-Hong Hou and P.R. Kumar 1.
Scheduling Periodic Real-Time Tasks with Heterogeneous Reward Requirements I-Hong Hou and P.R. Kumar 1 Presenter: Qixin Wang.
EECS 262a Advanced Topics in Computer Systems Lecture 13 Resource allocation: Lithe/DRF October 16 th, 2012 John Kubiatowicz and Anthony D. Joseph Electrical.
Using Map-reduce to Support MPMD Peng
1 Iterative Integer Programming Formulation for Robust Resource Allocation in Dynamic Real-Time Systems Sethavidh Gertphol and Viktor K. Prasanna University.
Towards Economic Fairness for Big Data Processing in Pay-as-you-go Cloud Computing Nanyang Technological University Shanjiang Tang, Bu-Sung Lee, Bingsheng.
MROrder: Flexible Job Ordering Optimization for Online MapReduce Workloads School of Computer Engineering Nanyang Technological University 30 th Aug 2013.
2002 Microeconomics Question 3.
Shanjiang Tang, Bu-Sung Lee, Bingsheng He, Haikun Liu School of Computer Engineering Nanyang Technological University Long-Term Resource Fairness Towards.
Dynamic Slot Allocation Technique for MapReduce Clusters School of Computer Engineering Nanyang Technological University 25 th Sept 2013 Shanjiang Tang,
June 30 - July 2, 2009AIMS 2009 Towards Energy Efficient Change Management in A Cloud Computing Environment: A Pro-Active Approach H. AbdelSalamK. Maly.
DynamicMR: A Dynamic Slot Allocation Optimization Framework for MapReduce Clusters Nanyang Technological University Shanjiang Tang, Bu-Sung Lee, Bingsheng.
© 2008 Pearson Education Canada22.1 Chapter 22 The ISLM Model.
Multi-Resource Packing for Cluster Schedulers Robert Grandl Aditya Akella Srikanth Kandula Ganesh Ananthanarayanan Sriram Rao.
Econ 2610: Principles of Microeconomics Yogesh Uppal
Using Map-reduce to Support MPMD Peng
John Kubiatowicz and Anthony D. Joseph
Farid Abolhassani Markets and Efficiency 10. Learning Objectives After working through this chapter, you will be able to: List and describe the assumptions.
Presented by Qifan Pu With many slides from Ali’s NSDI talk Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, Ion Stoica.
Dominant Resource Fairness: Fair Allocation of Multiple Resource Types Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, Ion.
Part III BigData Analysis Tools (YARN) Yuan Xue
1 Performance Impact of Resource Provisioning on Workflows Gurmeet Singh, Carl Kesselman and Ewa Deelman Information Science Institute University of Southern.
Chapter 7 Scheduling Chien-Chung Shen CIS/UD
Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center NSDI 11’ Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D.
CPU Scheduling Andy Wang Operating Systems COP 4610 / CGS 5765.
Scalable containers with Apache Mesos and DC/OS
Applied Operating System Concepts
CPU Scheduling CSSE 332 Operating Systems
Managerial Uses of Price Elasticity of Demand
Running Multiple Schedulers in Kubernetes
Lottery Scheduling and Dominant Resource Fairness (Lecture 24, cs262a)
CS 425 / ECE 428 Distributed Systems Fall 2016 Nov 10, 2016
Warm-Up Draw a correctly-labeled firm in monopolistic competition in long-run equilibrium. Mark the equilibrium P and Q for both monopolistic and perfect.
CS 425 / ECE 428 Distributed Systems Fall 2017 Nov 16, 2017
Perfect Competition No external parties (such as the Government) regulate the price, quantity, or quality of any of the goods being bought and sold in.
Summary Notes Economics 230 Part II.
Omega: flexible, scalable schedulers for large compute clusters
Andy Wang Operating Systems COP 4610 / CGS 5765
Measuring Costs and Benefits
COS 518: Advanced Computer Systems Lecture 14 Michael Freedman
Shanjiang Tang1, Bingsheng He2, Shuhao Zhang2,4, Zhaojie Niu3
The Price Adjustment Process
Presentation transcript:

MESOS

MESOS BEHAVIOR Framework rampup time: time it takes a new framework to achieve all of its allocation( expectation of time or maximum time) Job completion time: time it takes a job to complete, assuming one job per framework. System utilization: total cluster utilization

ELASTIC & CONST: each task takes T to finish ELASTIC & CONST: each task takes T to finish. The framework when allocated k slots at once, will take bT time to finish. Ramp up time: At most after T time, all k slots will be available. Compeletion time: When ramping up, on average kT/2 computation time is lost.

ELASTIC & EXP: each task takes T to finish ELASTIC & EXP: each task takes T to finish. The framework when allocated k slots at once, will take bT time to finish. Before allocating, the slots originally has tasks running, the expecatation of finishing time is T and they follow k i.i.d of exp distribution. Ramp up time: On avearge after Tlnk time, all k slots will be available. Compeletion time: When ramping up, on average kT computation time is lost.

RIGID & CONST: each task takes T to finish RIGID & CONST: each task takes T to finish. The framework when allocated k slots at once, will take bT time to finish. Ramp up time: At most after T time, all k slots will be available. Compeletion time: When ramping up, at most kT computation time is lost. Utlization: Consider actual total computation time/ total computation time when slots are busy

Compeletion time: When ramping up, at most kT computation time is lost. Utlization: Consider actual total computation time/ total computation time when slots are busy. When ramping up, the average computation time wasted is kT/2

RIGID & EXP: each task takes T to finish RIGID & EXP: each task takes T to finish. The framework when allocated k slots at once, will take bT time. Compeletion time: When ramping up, on average Tlnk computation time is lost. Utlization: Consider actual total computation time/ total computation time when slots are busy

DOMINANT RESOURCE FAIRNESS

Important properties

Dominant resource fairness Fairness definiton: Consider a cluster with (9 CPU, 18GB RAM). Job A runs tasks with demand vector(1, 4), Job B runs tasks with demand vector(3, 1).

Comparison with other fair allocation policies Asset fairness: 9 CPU, 18GB RAM. Then 1 CPU is worth $2, 1GB RAM is worth $1. Consider the same example jobs, Job A spends $6 each task, Job B spends $7 each tasks. Competitive equilibrium from equal incomes:

Result

Analysis