23 September 2004 Evaluating Adaptive Middleware Load Balancing Strategies for Middleware Systems Department of Electrical Engineering & Computer Science.

Slides:



Advertisements
Similar presentations
Module 13: Performance Tuning. Overview Performance tuning methodologies Instance level Database level Application level Overview of tools and techniques.
Advertisements

Performance Testing - Kanwalpreet Singh.
Distributed Systems Major Design Issues Presented by: Christopher Hector CS8320 – Advanced Operating Systems Spring 2007 – Section 2.6 Presentation Dr.
Database Architectures and the Web
Efficient Event-based Resource Discovery Wei Yan*, Songlin Hu*, Vinod Muthusamy +, Hans-Arno Jacobsen +, Li Zha* * Chinese Academy of Sciences, Beijing.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 9 Distributed Systems Architectures Slide 1 1 Chapter 9 Distributed Systems Architectures.
Lecturer: Sebastian Coope Ashton Building, Room G.18 COMP 201 web-page: Lecture.
Walter Binder University of Lugano, Switzerland Niranjan Suri IHMC, Florida, USA Green Computing: Energy Consumption Optimized Service Hosting.
Trace Analysis Chunxu Tang. The Mystery Machine: End-to-end performance analysis of large-scale Internet services.
Net-Centric Software and Systems I/UCRC Copyright © 2011 NSF Net-Centric I/UCRC. All Rights Reserved. High-Confidence SLA Assurance for Cloud Computing.
Technical Architectures
Distributed Systems Architectures
Quality of Service in IN-home digital networks Alina Albu 7 November 2003.
CS 501: Software Engineering Fall 2000 Lecture 16 System Architecture III Distributed Objects.
Software Engineering and Middleware: a Roadmap by Wolfgang Emmerich Ebru Dincel Sahitya Gupta.
Investigating Lightweight Fault Tolerance Strategies for Enterprise Distributed Real-time Embedded Systems Tech-X Corporation Boulder, Colorado Vanderbilt.
1 Dynamic CPU Management for Real-Time, Middleware-Based Systems Eric EideTim Stack Eric EideTim Stack John RegehrJay Lepreau University of Utah, School.
1 Introduction to Load Balancing: l Definition of Distributed systems. Collection of independent loosely coupled computing resources. l Load Balancing.
Adaptive Content Delivery for Scalable Web Servers Authors: Rahul Pradhan and Mark Claypool Presented by: David Finkel Computer Science Department Worcester.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 12 Slide 1 Distributed Systems Design 1.
New Challenges in Cloud Datacenter Monitoring and Management
The Design and Performance of A Real-Time CORBA Scheduling Service Christopher Gill, David Levine, Douglas Schmidt.
Self-Adaptive QoS Guarantees and Optimization in Clouds Jim (Zhanwen) Li (Carleton University) Murray Woodside (Carleton University) John Chinneck (Carleton.
Load Balancing Dan Priece. What is Load Balancing? Distributed computing with multiple resources Need some way to distribute workload Discreet from the.
©Ian Sommerville 2006Software Engineering, 8th edition. Chapter 12 Slide 1 Distributed Systems Architectures.
Database Architectures and the Web Session 5
26 Sep 2003 Transparent Adaptive Resource Management for Distributed Systems Department of Electrical Engineering and Computer Science Vanderbilt University,
Managing Service Metadata as Context The 2005 Istanbul International Computational Science & Engineering Conference (ICCSE2005) Mehmet S. Aktas
18 June 2001 Optimizing Distributed System Performance via Adaptive Middleware Load Balancing Ossama Othman Douglas C. Schmidt
Application-Layer Anycasting By Samarat Bhattacharjee et al. Presented by Matt Miller September 30, 2002.
Cluster Reliability Project ISIS Vanderbilt University.
Wireless Access and Terminal Mobility in CORBA Dimple Kaul, Arundhati Kogekar, Stoyan Paunov.
Real-Time Performance and Middleware for Multiprocessor and Multicore Linux Platforms* Yuanfang Zhang, Christopher Gill, and Chenyang Lu Department of.
Composing Adaptive Software Authors Philip K. McKinley, Seyed Masoud Sadjadi, Eric P. Kasten, Betty H.C. Cheng Presented by Ana Rodriguez June 21, 2006.
HPEC’02 Workshop September 24-26, 2002, MIT Lincoln Labs Applying Model-Integrated Computing & DRE Middleware to High- Performance Embedded Computing Applications.
Marcelo R.N. Mendes. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;
BOF: Megajobs Gracie: Grid Resource Virtualization and Customization Infrastructure How to execute hundreds of thousands tasks concurrently on distributed.
Decision-Theoretic Planning with (Re)Deployment of Components in Distributed Real-time & Embedded Systems Douglas C. Schmidt
Issues Autonomic operation (fault tolerance) Minimize interference to applications Hardware support for new operating systems Resource management (global.
Investigating Survivability Strategies for Ultra-Large Scale (ULS) Systems Vanderbilt University Nashville, Tennessee Institute for Software Integrated.
GVis: Grid-enabled Interactive Visualization State Key Laboratory. of CAD&CG Zhejiang University, Hangzhou
CS 501: Software Engineering Fall 1999 Lecture 12 System Architecture III Distributed Objects.
A Utility-based Approach to Scheduling Multimedia Streams in P2P Systems Fang Chen Computer Science Dept. University of California, Riverside
Request Distribution in Server Clusters Krithi Ramamritham Indian Institute of Technology Bombay.
Embedded System Lab. 정범종 A_DRM: Architecture-aware Distributed Resource Management of Virtualized Clusters H. Wang et al. VEE, 2015.
NetQoPE: A Middleware-based Netowork QoS Provisioning Engine for Distributed Real-time and Embedded Systems Jaiganesh Balasubramanian
Resources Management and Component Placement Presenter:Bo Sheng.
Integration of QoS-enabled Distributed Object Computing Middleware for Developing Next- Generation Distributed Applications By Krishnamurthy et Al. Presented.
Adaptive Resource Management Architecture for DRE Systems Nishanth Shankaran
1 BBN Technologies Quality Objects (QuO): Adaptive Management and Control Middleware for End-to-End QoS Craig Rodrigues, Joseph P. Loyall, Richard E. Schantz.
A Grid-enabled Multi-server Network Game Architecture Tianqi Wang, Cho-Li Wang, Francis C.M.Lau Department of Computer Science and Information Systems.
Marcelo R.N. Mendes. What is FINCoS? A set of tools for data generation, load submission, and performance measurement of CEP systems; Main Characteristics:
Distributed System Architectures Yonsei University 2 nd Semester, 2014 Woo-Cheol Kim.
Topic 2: The Role of Open Standards, Open-Source Development, & Different Development Models & Processes (on Industrializing Software) ARO Workshop Outbrief,
GRID ANATOMY Advanced Computing Concepts – Dr. Emmanuel Pilli.
CSC 480 Software Engineering Lecture 17 Nov 4, 2002.
Marcelo R.N. Mendes. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;
FLARe: a Fault-tolerant Lightweight Adaptive Real-time Middleware for Distributed Real-time and Embedded Systems Dr. Aniruddha S. Gokhale
Resource Optimization for Publisher/Subscriber-based Avionics Systems Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee.
Distributed Systems Architectures Chapter 12. Objectives  To explain the advantages and disadvantages of different distributed systems architectures.
Distributed Systems Architectures. Topics covered l Client-server architectures l Distributed object architectures l Inter-organisational computing.
Introduction to Load Balancing:
DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S
CSC 480 Software Engineering
Transparent Adaptive Resource Management for Middleware Systems
Ch > 28.4.
Distributed Systems Bina Ramamurthy 11/30/2018 B.Ramamurthy.
Load Balancing in Distributed Systems
Transparent Adaptive Resource Management for Middleware Systems
Presentation transcript:

23 September 2004 Evaluating Adaptive Middleware Load Balancing Strategies for Middleware Systems Department of Electrical Engineering & Computer Science Vanderbilt University, Nashville Jaiganesh Balasubramanian Ossama Othman Douglas C. Schmidt Lawrence Dowdy 8 th EDOC Conference, Monterey, California

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University R&D Motivation: Scaleable Distributed Systems Heterogeneous environments Large number of bursty clients Stringent QoS requirements, e.g.: –24x7 availability –Low latency & high throughput Examples –Online trading systems –Mission-critical systems for critical infrastructure e.g., air transportation, power grid control Our R&D goal is to assure the scalability of these types of distributed systems

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University General Approach: Scalability via Load Balancing Goal: Improve the efficiency & scalability of distributed systems Approach: Partition & balance application tasks across many computers via load distribution strategies Load balancing be implemented at multiple layers, e.g.: –Network layer –OS layer –Middleware layer

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Our Approach: Middleware Load Balancing Capabilities of middleware –Control end-to-end resources & QoS –Leverage hardware/software advances –Handle new environments & requirements –Provide reusable COTS services Motivation for middleware load balancing –Can take into account distributed system state, system runtime behavior, & request content –Application-level control over load balancing policies Hardware Domain-Specific Middleware Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware Operating Systems & Protocols Applications

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University TAO & Cygnus: Our Middleware Environment Based on The ACE ORB (TAO) – an open-source, high-performance, real-time CORBA Object Request Broker –deuce.doc.wustl.edu/ Download.html Cygnus is a middleware load balancing service implemented atop TAO TAO also provides many other middleware services LOADBALANCING RT-CORBA EVENT SERVICESECURITYTRANSACTIONS DYNAMIC/STATIC SCHEDULING AV STREAMS

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Cygnus Load Balancing Architecture Multiple Clients can invoke requests –Potentially non-deterministic –Duration called a session Members –Multiple instances of the same object implementation Object Groups –Collections of members among which loads will be distributed equitably Load Balancer –Transparently distributes requests to members within an object group

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Load Monitor –Provides load feedback Load Analyzer –Determines location and member load conditions Member Locator –Binds client to appropriate object group member Load Alert –Facilitates load control (load shedding) Load Manager –Mediates all interactions between load balancer components Cygnus Load Balancing Components

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Types of Load Balancing Strategies Non-adaptive strategies –Use static information to make load balancing decisions e.g., cyclic selection of servers Adaptive strategies –Use dynamic system information to make load balancing decisions e.g., runtime host load information, CPU utilization –Have runtime parameters that must be set properly

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Cygnus Load Balancing Strategies Non-adaptive strategies –RoundRobin & Random strategies Adaptive strategies –LeastLoaded strategy rejects new sessions so that load is balanced RejectThreshold controls runtime rejection decision –LoadMinimum strategy migrates client sessions from one machine to another to balance load MigrationThreshold controls runtime migration decision

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Cygnus Adaptive Load Balancing Interactions Acceptance strategies Migration strategies

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Evaluating Load Balancing Strategies with LBPerf We use LBPerf to empirically determine appropriate load balancing strategies for applications with various workload characteristics Open-source tool suite for evaluating middleware load balancing strategies Can tune different configurations of middleware load balancing –e.g., systematically choose different load balancing strategies & runtime parameters associated with those strategies Evaluate strategies using different metrics –e.g., throughput, latency, CPU utilization, overhead, etc. Client code generated by LBPerf Server code generated by LBPerf Load balancer code with option to change strategies & run-time parameters Client workloads can vary!!

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Summary of Testbed & Experiments Emulab is a NSF-sponsored testbed for conducting large-scale network/system experiments – LBPerf experiments conducted with 16 clients & 32 servers RedHat Linux 7.1, Pentium III 850Mhz & 600Mhz machines LBPerf experiments run in Emulab –Overhead measurements –Throughput measurements –CPU utilization –Migration threshold values –Reject threshold values

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Overhead of Cygnus (1/2) CPU-intensive requests Single server Single-threaded clients generate CPU-intensive requests on one server Repeat experiment with & without load balancer Measure throughput, defined as # of requests processed per- second by server Without load balancer With load balancer

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Overhead of Cygnus (2/2) Single-threaded clients generate CPU-intensive requests on one server Repeat experiment with & without load balancer Measure throughput, defined as # of requests processed per- second by server Overhead of using Cygnus is negligible

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Load Balancing Effectiveness (1/3) Load balancer code with option to change strategies & run-time parameters CPU-intensive requests Single-threaded clients generate CPU- intensive requests on 4 servers Repeat experiment, choosing each of four strategies Measure –Throughput, defined as # of requests processed per- second by server –CPU utilization at each server

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Load Balancing Effectiveness (2/3) Single-threaded clients generate CPU- intensive requests on 4 servers Repeat experiment, choosing each of four strategies Measure –Throughput, defined as # of requests processed per- second by server –CPU utilization at each server Adaptive strategies Non-adaptive strategies Adaptive strategies behave better than non-adaptive strategies with non-uniform CPU intensive workloads

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Load Balancing Effectiveness (3/3) Adaptive strategies ensure uniform utilization of servers Single-threaded clients generate CPU- intensive requests on 4 servers Repeat experiment, choosing each of four strategies Measure –Throughput, defined as # of requests processed per- second by server –CPU utilization at each server

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Effect of Migration Threshold (1/2) Single threaded clients generate CPU- intensive requests on 4 servers Use LoadMinimum strategy Repeat experiments for different MigrationThreshold values Measure throughput, defined as # of requests processed per-second by server Load balancer code with LoadMinimum strategy Change MigrationThreshold values CPU-intensive requests

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Single threaded clients generate CPU- intensive requests on 4 servers Use LoadMinimum strategy Repeat experiments for different MigrationThreshold values Measure throughput, defined as # of requests processed per-second by server Migration’s not always effective Measuring Effect of Migration Threshold (2/2)

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Effect of Reject Threshold (1/2) Single threaded clients generate CPU- intensive requests on 4 servers Use LeastLoaded strategy Repeat experiments for different RejectThreshold values Measure throughput, defined as # of requests processed per-second by server Load balancer code with LeastLoaded strategy Change RejectThreshold values CPU-intensive requests

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Measuring Effect of Reject Threshold (2/2) Single threaded clients generate CPU- intensive requests on 4 servers Use LeastLoaded strategy Repeat experiments for different RejectThreshold values Measure throughput, defined as # of requests processed per-second by server Reject sessions to ensure QoS for admitted sessions

Balasubramanian et allEvaluating Load Balancing Strategies Vanderbilt University Concluding Remarks Cygnus adds little overhead for load balancing, while providing increased scalability & utilization Adaptive load balancing strategies can perform much better than non-adaptive strategies in presence of non-uniform loads Different run-time parameters associated with adaptive strategies must be tuned to perform well Future work: Devise self-adaptive load balancing architectures –Take QoS specifications as input –Tune run-time parameters dynamically to achieve those specifications