1 Incentive-Based Scheduling for Market-Like Computational Grids Lijuan Xiao, Yanmin Zhu, Member, IEEE, Lionel M. Ni, Fellow, IEEE, and Zhiwei Xu, Senior Member, IEEE Present by Ting-Wei, Chen

2 Index  Introduction  Problem Formulation  The Incentive-Based Scheduling Scheme  Performance Evaluation  Conclusions

3 Introduction (cont.)  Market-like computational grid’s Characteristic –Allow providers and consumers to make autonomous scheduling decisions –Have sufficient incentives to stay and play in the market  Formulate a intuition of optimizing incentives as dual-objective scheduling problem –Maximize the success rate of job execution –Minimize fairness deviation among resources

4 Introduction (cont.)  Present an incentive-based scheduling scheme –IB –Peer-to-peer decentralized scheduling framework –A set of local heuristic algorithms –Market instrument Job announcement Price Competition degree (CD)

5 Introduction (cont.)  Object –Make autonomous decisions –Producing a desirable emergent property in the grid system

6 Problem Formulation (cont.)  Define a market-like computational grid  Four-tuple G=(R, S, J, M)

7 Problem Formulation (cont.)  Consumers and jobs –Only consider computation-intensive jobs –Job announcement Job length Job deadline  Providers and resources –Be modeled with three parameters Capability Job queue Unit price

8 Problem Formulation (cont.)  Incentives for consumers and providers –High quality of computational service at low cost –High successful-execution rate of jobs –Successful-execution rate θ 1, if 0, if

9 Problem Formulation (cont.)  Fairness deviation σ of the grid system where  Maximize θ  Minimize σ

10 The Incentive-Based Scheduling Scheme (cont.)  Characterized –Consumer or provider autonomously makes scheduling decisions –Scheduling algorithms are local to a resource provider –Three market instruments Job announcement Price CD

11 The Incentive-Based Scheduling Scheme (cont.)  Peer-to-Peer Scheduling Framework –Decentralization –Scalability –Dynamics of grid environments

12 The Incentive-Based Scheduling Scheme (cont.) –Computational grid G Via one of which a provider can join the grid Get the information of designated neighbors Connect into the P2P network –Consumer Submit a job announcement to the computational grid Spread throughout the P2P network Provider receive a job announcement

13 The Incentive-Based Scheduling Scheme (cont.) –Realize the complete competition All providers should have an equal chance to compete for any job The number of providers will not be too large –Solve the Blind-flooding-based broadcasting’s problem Building overlay networks Efficient broadcasting mechanism

14 The Incentive-Based Scheduling Scheme (cont.)

15 The Incentive-Based Scheduling Scheme (cont.)  Incentive-Based Scheduling Algorithms –Job competing algorithm Provider receives a job announcement

16 The Incentive-Based Scheduling Scheme (cont.) Step 1 The provider estimates whether it is able to meet the job deadline 1 if T L > T A then 2 can meet ←true; 3 reordered ← false; 4 insert place ← P q ; 5 else // T L is covered by the execution of J i in the queue 6 if insert s at P i-1, none of J i ~J q will miss its deadline then 7 can meet ← true; 8 reordered ← true; 9 insert place ← P i-1 ; 10 else 11 can meet ← false; 12 endif 13 endif

The Incentive-Based Scheduling Scheme (cont.) Step 2 The provider offers a price for the job Step 3 The provider sends the price as a bid and inserts the job at the place that the variable insert_place indicates at the probability of 1-CD price ← p*L s ; 2 if reordered then 3 price ← λ*price; 4 endif

18 The Incentive-Based Scheduling Scheme (cont.) –Heuristic Local Scheduling Algorithm Punishment mechanism How much time the completion time T C exceeds the deadline T D Every time a provider is offered a job that is not kept in the job queue

The Incentive-Based Scheduling Scheme (cont.) 1 insert place ← P q ; 2 penalty ← calculate the penalty of inserting the job at P q ; 3 for i ← q-1 to 0 do 4 penalty i ← calculate the penalty of inserting the job at P i ; 5 if penalty i < penalty then 6 penalty ← penalty i ; 7 insert_place ← P i ; 8 endif 9 endfor 10 insert the job at insert_place 19

The Incentive-Based Scheduling Scheme (cont.) –Price-Adjusting Algorithm Price mechanism –Make prices different –Differentiate the chances of providers to be chosen –Eventually realize the fair allocation of profits All the providers need to know some global information Every time a provider is offered a job or deletes an unconfirmed job, then start the price-adjusting algorithm The price will fluctuate around the market price 20

The Incentive-Based Scheduling Scheme (cont.) 1 r1 ← L O /L T ; 2 r2 ← C/∑ 0≤j<m C j ; 3 if offered a job then 4 if r1 > r2 and p <=P M then 5 p ← α*p; 6 endif 7 else // delete an unconfirmed job 8 if r1 =P M then 9 p ← β*p; 10 endif 11 endif 21

The Incentive-Based Scheduling Scheme (cont.) –Competition-Degree-Adjusting Algorithm Keep unconfirmed jobs in their job queues 22 //Every time the penalty increases 1 if R p >=TH p and CD >=ε then 2 CD ← CD - ε; 3 endif //Every time a certain interval such as 1 day 1 if R p =TH J and CD <=1- ε then 2 CD ← CD+ ε; 3 endif

23 Performance Evaluation (cont.)  Evaluation Methodology –Discrete event-driven simulator –Mainly drive: The network delay of communication (Ignore) Job execution –Four experiments The impact of CD on performance Analyzes the incentive-based scheduling scheme by disabling the CD-adjusting algorithm Compare IB, FCFS, SJF, EDF, and FirstReward under synthetic workloads and real workloads

24 Performance Evaluation (cont.) –System load Over a period of time T is defined as the ratio of the aggregated length of jobs submitted to the aggregated job length that the computational grid is capable to execute. System load varies from 0.1 to 0.7

Performance Evaluation (cont.) –Consumers Job generation is modeled as a Poisson process The deadline is uniformly distributed as well –Providers Distributed on account of the observation Predominate in computer market 25

26 Performance Evaluation (cont.)  Four experiment –Impact of CD on performance Failure Rates of Jobs

27 Performance Evaluation (cont.) Deadline Missing Rates of Jobs The successful-execution rate θ can be calculated

28 Performance Evaluation (cont.) Successful-execution rates of jobs –The conservative attitude (CD=0) toward competing for jobs is nota desirable one, considering the successful-execution rate.

29 Performance Evaluation (cont.) The Total Penalty of Providers

Performance Evaluation (cont.) Total profit of providers

31 Performance Evaluation (cont.) –Analyzes the incentive-based scheduling scheme Analysis of IB on the successful-execution rate

32 Performance Evaluation (cont.) –Compare our scheme with four other schemes under synthetic workloads FCFS SJF EDF FirstReward IB

33 Performance Evaluation (cont.) Comparison on the successful-execution rate

34 Performance Evaluation (cont.) Comparison on the fairness deviation

35 Performance Evaluation (cont.) –Compare our scheme with four other schemes under real workload Chose the LPC EGEE (Laboratoire de Physique Corpusculaire Enabling Grids for E-sciencE) trace

36 Performance Evaluation (cont.) Fair profit allocation of IB versus unfair profit allocation of EDF

37 Performance Evaluation (cont.) Balanced utilization of IB versus imbalanced utilization of EDF

38 Conclusion (cont.)  Incentive-based scheduling scheme IB feature –Consumer and provider autonomously makes scheduling decisions –All scheduling algorithms are local to a resource provider –Three market instruments

39 Conclusion (cont.)  Advantages –Participant makes local/autonomous decision –High successful-job-execution rate –Fair allocation of profits –Balanced utilization of resource

40 Thank you for your attention