FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor.

Slides:

Advertisements

Similar presentations

Libra: An Economy driven Job Scheduling System for Clusters Jahanzeb Sherwani 1, Nosheen Ali 1, Nausheen Lotia 1, Zahra Hayat 1, Rajkumar Buyya 2 1. Lahore.

Advertisements

Pricing for Utility-driven Resource Management and Allocation in Clusters Chee Shin Yeo and Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS)

New Micro Genetic Algorithm for multi-user detection in WCDMA AZMI BIN AHMAD Borhanuddin Mohd Ali, Sabira Khatun, Azmi Hassan Dept of Computer and Communication.

CSF4 Meta-Scheduler Tutorial 1st PRAGMA Institute Zhaohui Ding or

CSF4 Meta-Scheduler PRAGMA13 Zhaohui Ding or College of Computer.

Teacher Name Class / Subject Date A:B: Write an answer here #1 Write your question Here C:D: Write an answer here.

CS4026 Formal Models of Computation Running Haskell Programs – power.

Dynamic Server Allocation in Heterogeneous Clusters J. Palmer I. Mitrani School of Computing Science University of Newcastle NE1 7RU

© Actility – Confidential – Under NDA 1 Advanced flexibility management: concepts and opportunities Making Things Smart.

Flashcards? Lets Get Started. Question 1 Get Answer.

Duagi Bulent UNIVERSITY POLITEHNICA of BUCHAREST DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY POLITEHNICA of BUCHAREST DEPARTMENT OF COMPUTER SCIENCE.

IGLS/1 © P. Pongcharoen Using Genetic Algorithms for Scheduling the Production of Capital Goods P. Pongcharoen, C. Hicks, P.M. Braiden, A.V. Metcalfe,

Energy-efficient Task Scheduling in Heterogeneous Environment 2013/10/25.

5.9 + = 10 a)3.6 b)4.1 c)5.3 Question 1: Good Answer!! Well Done!! = 10 Question 1:

Devising Secure Sockets Layer-Based Distributed Systems: A Performance-Aware Approach Norman Lim, Shikharesh Majumdar,Vineet Srivastava, Dept. of Systems.

FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES Interference-Aware Message Forwarding for Vehicular Networks Centre for Distributed and High Performance.

Processes Management.

1 Zi Yang, Wei Li, Jie Tang, and Juanzi Li Knowledge Engineering Group Department of Computer Science and Technology Tsinghua University, China {yangzi,

Presented by: Priti Lohani

Academic skills… Using information from sources  Claims and evidence  Quoting, summarising and paraphrasing  Referencing © University of Sydney Learning.

Adaptive Scheduling with QoS Satisfaction in Hybrid Cloud Environment 研究生：李羿慷指導老師：張玉山老師.

Fuzzy Simulated Evolution for Power and Performance of VLSI Placement Sadiq M. Sait Habib Youssef Junaid A. KhanAimane El-Maleh Department of Computer.

A Grid Resource Broker Supporting Advance Reservations and Benchmark- Based Resource Selection Erik Elmroth and Johan Tordsson Reporter ： S.Y.Chen.

4/26/05Han: ELEC72501 Department of Electrical and Computer Engineering Auburn University, AL K.Han Development of Parallel Distributed Computing System.

Optimization for QoS on Systems with Tasks Deadlines Luis Fernando Orleans Pedro Nuno Furtado.

Fault-tolerant Adaptive Divisible Load Scheduling Xuan Lin, Sumanth J. V. Acknowledge: a few slides of DLT are from Thomas Robertazzi ’ s presentation.

Security-Driven Heuristics and A Fast Genetic Algorithm for Trusted Grid Job Scheduling Shanshan Song, Ricky Kwok, and Kai Hwang University of Southern.

Grid Load Balancing Scheduling Algorithm Based on Statistics Thinking The 9th International Conference for Young Computer Scientists Bin Lu, Hongbin Zhang.

1 TDMA Scheduling in Competitive Wireless Networks Mario CagaljHai Zhan EPFL - I&C - LCA February 9, 2005.

On Fairness, Optimizing Replica Selection in Data Grids Husni Hamad E. AL-Mistarihi and Chan Huah Yong IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,

1 A Cooperative Game Framework for QoS Guided Job Allocation Schemes in Grids Riky Subrata, Member, IEEE, Albert Y. Zomaya, Fellow, IEEE, and Bjorn Landfeldt,

Distributed computation and parameter estimation on identification of physiological systems Tomáš Kulhánek 1,2 Jan Šilar 1 Marek Mateják 1 Pavol Privitzer.

A Budget Constrained Scheduling of Workflow Applications on Utility Grids using Genetic Algorithms Jia Yu and Rajkumar Buyya Grid Computing and Distributed.

Using Grid Computing in Parallel Electronic Circuit Simulation Marko Dimitrijević FACULTY OF ELECTRONIC ENGINEERING, UNIVERSITY OF NIŠ LABORATORY FOR ELECTRONIC.

Resource Provisioning based on Lease Preemption in InterGrid Mohsen Amini Salehi, Bahman Javadi, Rajkumar Buyya Cloud Computing and Distributed Systems.

Computer Science and Engineering Parallel and Distributed Processing CSE 8380 March 01, 2005 Session 14.

1 A Framework for Data-Intensive Computing with Cloud Bursting Tekin Bicer David ChiuGagan Agrawal Department of Compute Science and Engineering The Ohio.

A Survey of Distributed Task Schedulers Kei Takahashi (M1)

ASC2003 (July 15,2003)1 Uniformly Distributed Sampling: An Exact Algorithm for GA’s Initial Population in A Tree Graph H. S.

Autonomic scheduling of tasks from data parallel patterns to CPU/GPU core mixes Published in: High Performance Computing and Simulation (HPCS), 2013 International.

E-science grid facility for Europe and Latin America E2GRIS1 Gustavo Miranda Teixeira Ricardo Silva Campos Laboratório de Fisiologia Computacional.

LOGO Development of the distributed computing system for the MPD at the NICA collider, analytical estimations Mathematical Modeling and Computational Physics.

2/29/20121 Optimizing LCLS2 taper profile with genetic algorithms: preliminary results X. Huang, J. Wu, T. Raubenhaimer, Y. Jiao, S. Spampinati, A. Mandlekar,

Performance Analysis of Preemption-aware Scheduling in Multi-Cluster Grid Environments Mohsen Amini Salehi, Bahman Javadi, Rajkumar Buyya Cloud Computing.

Economic and On Demand Brain Activity Analysis on Global Grids A case study.

Grid Appliance The World of Virtual Resource Sharing Group # 14 Dhairya Gala Priyank Shah.

Scheduling MPI Workflow Applications on Computing Grids Juemin Zhang, Waleed Meleis, and David Kaeli Electrical and Computer Engineering Department, Northeastern.

Xi He Golisano College of Computing and Information Sciences Rochester Institute of Technology Rochester, NY THERMAL-AWARE RESOURCE.

Evolving RBF Networks via GP for Estimating Fitness Values using Surrogate Models Ahmed Kattan Edgar Galvan.

Data Consolidation: A Task Scheduling and Data Migration Technique for Grid Networks Author: P. Kokkinos, K. Christodoulopoulos, A. Kretsis, and E. Varvarigos.

Developing resource consolidation frameworks for moldable virtual machines in clouds Author: Liang He, Deqing Zou, Zhang Zhang, etc Presenter: Weida Zhong.

Grid technologies for large-scale projects N. S. Astakhov, A. S. Baginyan, S. D. Belov, A. G. Dolbilov, A. O. Golunov, I. N. Gorbunov, N. I. Gromova, I.

Distributed Correlation in Fabric Kiwi Team PSNC.

Igor EPIMAKHOV Abdelkader HAMEURLAIN Franck MORVAN

Simulation and Exploration of

Deadline Scheduling and Heavy tail distributionS

Mean Value Analysis of a Database Grid Application

A Dynamic Critical Path Algorithm for Scheduling Scientific Workflow Applications on Global Grids e-Science IEEE 2007 Report: Wei-Cheng Lee

Bin Packing Optimization

PhD student, Industrial & Manufacturing Engineering, UW-Milwaukee

Introducing – SAS® Grid Manager for Hadoop

Physics-based simulation for visual computing applications

Mike Becher and Wolfgang Rehm

Scheduling Jobs in Multi-Grid Environment

Execution Framework: Hadoop 2.x

ANALYSIS OF USER SUBMISSION BEHAVIOR ON HPC AND HTC

User interaction and workflow management in Grid enabled e-VLBI experiments Dominik Stokłosa Poznań Supercomputing and Networking Center, Supercomputing.

Estimation Method of Moments Industrial Engineering

Stochastic Simulations

Approximate Mean Value Analysis of a Database Grid Application

Presentation transcript:

FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor and Director Centre for Distributed and High Performance Computing School of Information Technologies The University of Sydney, Sydney, Australia Javid Taheri | Postdoctoral Research Fellow

›Introduction to Grid Computing ›Problem Statement: Data-Aware Job Scheduling ›GA-ParFnt -Pareto Frontier -Genetic Algorithm (GA) ›Simulation and Analysis of Results ›Conclusion 2

Grid Computing 3

Problem Statement ›Data Aware Job Scheduling (DAJS) -(1) the overall execution time of a batch of jobs (NP-Complete) -(2) transfer time of all datafiles to their dependent jobs (NP-Complete) 4 Storage Nodes Storage Nodes Computation Nodes Computation Nodes Job 1 Job 2 Job 3 Job N... File 1 File 2 File 3... File M

Problem Statement (cont.) 5 SN CN Scheduler

Preliminaries ›Pareto Front ›Genetic Algorithm 6

GA for Finding DAJS’ Pareto Front (GA-ParFnt) 7

Simulation ›Test-Grid-4-8 8

Discussion and Analysis ›The shape of Pareto Front 9 Test-Grid-8-4

Discussion and Analysis ›Scheduling Algorithms 10

Conclusion ›GA-ParFnt was effective in finding the Pareto Front of executing jobs vs Transfer time of Datafiles in Grids ›Such Pareto Front could be estimated by exponential funcitons ›Many scheduling algorithms are not optimal, despite their claim. 11

THANK YOU Questions? 12