CS433/533 Computer Networks Lecture 12 Load Balancing Networks 2/16/2012 1.

Slides:



Advertisements
Similar presentations
CSC458 Programming Assignment II: NAT Nov 7, 2014.
Advertisements

CPSC Network Layer4-1 IP addresses: how to get one? Q: How does a host get IP address? r hard-coded by system admin in a file m Windows: control-panel->network->configuration-
© 2007 Cisco Systems, Inc. All rights reserved.Cisco Public 1 Addressing the Network – IPv4 Network Fundamentals – Chapter 6.
Cs/ee 143 Communication Networks Chapter 6 Internetworking Text: Walrand & Parekh, 2010 Steven Low CMS, EE, Caltech.
Implementing Inter-VLAN Routing
Switching Topic 4 Inter-VLAN routing. Agenda Routing process Routing VLANs – Traditional model – Router-on-a-stick – Multilayer switches EtherChannel.
SYSTEM ADMINISTRATION Chapter 19
MCDST : Supporting Users and Troubleshooting a Microsoft Windows XP Operating System Chapter 13: Troubleshoot TCP/IP.
Module 8: Concepts of a Network Load Balancing Cluster
1 Internet Networking Spring 2004 Tutorial 13 LSNAT - Load Sharing NAT (RFC 2391)
Page: 1 Director 1.0 TECHNION Department of Computer Science The Computer Communication Lab (236340) Summer 2002 Submitted by: David Schwartz Idan Zak.
Chapter Extension 7 How the Internet Works © 2008 Prentice Hall, Experiencing MIS, David Kroenke.
1 Improving Web Servers performance Objectives:  Scalable Web server System  Locally distributed architectures  Cluster-based Web systems  Distributed.
Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 1 Unit Outline Information.
Lesson 1: Configuring Network Load Balancing
1 Spring Semester 2007, Dept. of Computer Science, Technion Internet Networking recitation #12 LSNAT - Load Sharing NAT (RFC 2391)
(part 3).  Switches, also known as switching hubs, have become an increasingly important part of our networking today, because when working with hubs,
Connecting LANs, Backbone Networks, and Virtual LANs
Firewalls CS432. Overview  What are firewalls?  Types of firewalls Packet filtering firewalls Packet filtering firewalls Sateful firewalls Sateful firewalls.
Chapter 5 Link Layer Computer Networking: A Top Down Approach 6 th edition Jim Kurose, Keith Ross Addison-Wesley March 2012 Link Layer data center.
Chapter Eleven An Introduction to TCP/IP. Objectives To compare TCP/IP’s layered structure to OSI To review the structure of an IP address To look at.
CN2668 Routers and Switches Kemtis Kunanuraksapong MSIS with Distinction MCTS, MCDST, MCP, A+
© 2007 Cisco Systems, Inc. All rights reserved.Cisco Public 1 Addressing the Network – IPv4 Network Fundamentals – Chapter 6.
CECS 5460 – Assignment 3 Stacey VanderHeiden Güney.
CS433/533 Computer Networks Lecture 12 Load Balancing Networks 10/8/
23-Support Protocols and Technologies Dr. John P. Abraham Professor UTPA.
Module 13: Network Load Balancing Fundamentals. Server Availability and Scalability Overview Windows Network Load Balancing Configuring Windows Network.
70-291: MCSE Guide to Managing a Microsoft Windows Server 2003 Network Chapter 3: TCP/IP Architecture.
DNS (Domain Name System) Protocol On the Internet, the DNS associates various sorts of information with domain names. A domain name is a meaningful and.
© 2006 Cisco Systems, Inc. All rights reserved.1 Microsoft Network Load Balancing Support Vivek V
PA3: Router Junxian (Jim) Huang EECS 489 W11 /
Implementing ISA Server Publishing. Introduction What Are Web Publishing Rules? ISA Server uses Web publishing rules to make Web sites on protected networks.
Network Admin Course Plan Accede Institute Of Science & Technology.
Cisco – Chapter 11 Routers All You Ever Wanted To Know But Were Afraid to Ask.
CS433/533 Computer Networks Lecture 12 CDN 2/16/2012.
Objectives Configure routing in Windows Server 2008 Configure Routing and Remote Access Services in Windows Server 2008 Network Address Translation 1.
Examining TCP/IP.
CMPT 471 Networking II Address Resolution IPv4 ARP RARP 1© Janice Regan, 2012.
Module 3: Designing IP Addressing. Module Overview Designing an IPv4 Addressing Scheme Designing DHCP Implementation Designing DHCP Configuration Options.
NUS.SOC.CS2105 Ooi Wei Tsang Application Transport Network Link Physical you are here.
Clustering and Load Balancing. Outline Introduction Linux Virtual Server Microsoft load balancing solution.
Web Cache Redirection using a Layer-4 switch: Architecture, issues, tradeoffs, and trends Shirish Sathaye Vice-President of Engineering.
Review: –Ethernet What is the MAC protocol in Ethernet? –CSMA/CD –Binary exponential backoff Is there any relationship between the minimum frame size and.
Transport Layer3-1 Chapter 4: Network Layer r 4. 1 Introduction r 4.2 Virtual circuit and datagram networks r 4.3 What’s inside a router r 4.4 IP: Internet.
Homework 02 NAT 、 DHCP 、 Firewall 、 Proxy. Computer Center, CS, NCTU 2 Basic Knowledge  DHCP Dynamically assigning IPs to clients  NAT Translating addresses.
1 © 2003, Cisco Systems, Inc. All rights reserved. CCNA 3 v3.0 Module 9 Virtual Trunking Protocol.
Switch Features Most enterprise-capable switches have a number of features that make the switch attractive for large organizations. The following is a.
70-291: MCSE Guide to Managing a Microsoft Windows Server 2003 Network, Enhanced Chapter 3: TCP/IP Architecture.
The Client-Server Model And the Socket API. Client-Server (1) The datagram service does not require cooperation between the peer applications but such.
70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 4: Planning and Configuring Routing and Switching.
BNL PDN Enhancements. Perimeter Load Balancers Scaleable Performance Fault Tolerance Server Maintainability User Convenience Perimeter Security.
Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved. 1 Cisco Networking Training (CCENT/CCT/CCNA R&S) Rick Rowe Ron Giannetti.
Data Communications and Computer Networks Chapter 4 CS 3830 Lecture 19 Omar Meqdadi Department of Computer Science and Software Engineering University.
1 Objectives Identify the basic components of a network Describe the features of Internet Protocol version 4 (IPv4) and Internet Protocol version 6 (IPv6)
Chapter 5. An IP address is simply a series of binary bits (ones and zeros). How many binary bits are used? 32.
Address Resolution Protocol Yasir Jan 20 th March 2008 Future Internet.
CSC458 Programming Assignment II: NAT
Link Layer 5.1 Introduction and services
F5 BIGIP V 9 Training.
MAC Addresses and ARP 32-bit IP address:
VIRTUAL SERVERS Presented By: Ravi Joshi IV Year (IT)
Chapter 4 Data Link Layer Switching
ARP: Address Resolution Protocol
Routing and Switching Essentials v6.0
70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 4: Planning and Configuring Routing and Switching.
1 ADDRESS RESOLUTION PROTOCOL (ARP) & REVERSE ADDRESS RESOLUTION PROTOCOL ( RARP) K. PALANIVEL Systems Analyst, Computer Centre Pondicherry University,
Network Applications: Multi-Server Request Routing
Ch 17 - Binding Protocol Addresses
Computer Networks ARP and RARP
Chapter 5: Link Layer 5.1 Introduction and services
Presentation transcript:

CS433/533 Computer Networks Lecture 12 Load Balancing Networks 2/16/2012 1

Outline  Admin and recap  Load balance (direction) o Load direction mechanisms o DNS o Load balancer (smart switch) 2

Admin r Programming assignment 1 status 3

Recap: High-Performance Network Servers r Avoid blocking (so that we can reach bottleneck throughput) m threads r Limit unlimited thread overhead m thread pool, async io r Coordinating data access m synchronization (lock, synchronized) r Coordinating behavior: avoid busy-wait m wait/notify; FSM r Extensibility/robustness m language support/design for interfaces 4

Recap: Operational Laws r Utilization law: U = XS r Forced flow law: Xi = Vi X r Bottleneck device: largest Di = Vi Si r Little’s Law: Qi = Xi Ri r Bottleneck analysis: 5

Recap: Why Multiple Network Servers? r Scalability m beyond single server capability and geolocation of a single server r Redundancy and fault tolerance m administration/maintenance (e.g., incremental upgrade) m redundancy (e.g., to handle failures) r System/software architecture m resources may be naturally distributed at different machines (e.g., run a single copy of a database server due to single license; access to resource from third party) m security (e.g., front end, business logic, and database) 6

Recap: Load Direction: Basic Architecture r Major components m Server state monitoring m Network path properties between clients and servers m Server selection alg. m Server routing/adaptation m Client direction mechanism 7 Internet Client Site A Site B ?

Recap: Load Direction 8 server state net path property between servers/clients server selection algorithm specific request of a client notify client about selection (direction mech) server routing

Network Path Properties  Why is the problem difficult? o Scalability: if do measurements, complete measurements grow with N * M, where N is # of sources (e.g., clients) M is # of destinations (e.g., servers) o Complexity/feasibility in computing path metrics 9 M N

Network Path Properties: Improve Scalability  Aggregation: o merge a set of IP addresses (reduce N and M) E.g., when computing path properties, Akamai aggregates all clients sharing the same local DNS server  Sampling and prediction o Instead of measuring N*M entries, we measure a subset and predict the unmeasured paths o We will cover it later in the course 10 M servers N’~100K local DNS servers N~all Internet clients

Server Selection  Why is the problem difficult?  What are potential problems of just sending each new client to the lightest load server? 11

Outline  Admin and recap  Load balance (direction) o Overview o Basic load direction mechanisms 12

Basic Direction Mechanisms r Application layer m App/user is given a list of candidate server names m HTTP redirector r DNS: name resolution gives a list of server addresses r IP layer: Same IP address represents multiple physical servers m IP anycast: Same IP address shared by multiple servers and announced at different parts of the Internet. Network directs different clients to different servers (e.g., Limelight) m Load balancer (smart switch) indirection m Reverse proxy 13

Direction Mechanisms are Often Combined 14 DNS name1 IP1 IP2IPn Cluster1 in US East Cluster2 in US West Load balancer Load balancer proxy Cluster2 in Europe Load balancer Load balancer servers DNS name2

Example: Netflix 15

Example: Netflix Manifest File 16 r Client player authenticate and then downloads manifest file from servers at Amazon Cloud

Example: Netflix Manifest File 17

Example: Wikipedia Architecture 18

Outline  Admin and recap  Load balance (direction) o Basic load direction mechanisms o DNS 19

DNS Indirection and Rotation router DNS server for cnn.com IP address of cnn.com IP address of cnn.com

Example: Amazon Elastic Cloud 2 (EC2) Elastic Load Balancing r Use the elb-create-lb command to create an Elastic Load Balancer. r Use the elb-register-instances -with-lb command to register the Amazon EC2 instances that you want to load balance with the Elastic Load Balancer. r Elastic Load Balancing automatically checks the health of your load balancing Amazon EC2 instances. You can optionally customize the health checks by using the elb-configure-healthcheck command. r Traffic to the DNS name provided by the Elastic Load Balancer is automatically distributed across your load balanced, healthy Amazon EC2 instances. 21

Details: Step 1 1. Call CreateLoadBalancer with the following parameters: CreateLoadBalancer m AvailabilityZones = us-east-1a m Listeners Protocol = HTTP InstancePort = 8080 LoadBalancerPort = 80 LoadBalancerName = MyLoadBalancer The operation returns the DNS name of your LoadBalancer. You can then map that to any other domain name (such as (how?) PROMPT> elb-create-lb MyLoadBalancer --headers -- listener "lb-port=80,instance-port=8080,protocol=HTTP" - -availability-zones us-east-1a Result: DNS-NAME DNS-NAME DNS-NAME MyLoadBalancer us-east-1.elb.amazonaws.com 22

Details: Step 2 2. Call ConfigureHealthCheck with the following parameters:ConfigureHealthCheck m LoadBalancerName = MyLoadBalancer m Target = NoteMake sure your instances respond to /ping on port 8080 with an HTTP 200 status code. m Interval = 30 m Timeout = 3 m HealthyThreshold = 2 m UnhealthyThreshold = 2 PROMPT> elb-configure-healthcheck MyLoadBalancer --headers --target " --interval 30 --timeout 3 --unhealthy-threshold 2 -- healthy-threshold 2 Result: HEALTH-CHECK TARGET INTERVAL TIMEOUT HEALTHY-THRESHOLD UNHEALTHY-THRESHOLD HEALTH-CHECK

Details: Step 3 3. Call RegisterInstancesWithLoadBalancer with the following parameters:RegisterInstancesWithLoadBalancer m LoadBalancerName = MyLoadBalancer m Instances = [ i-4f8cf126, i-0bb7ca62 ] PROMPT> elb-register-instances-with-lb MyLoadBalancer --headers -- instances i-4f8cf126,i-0bb7ca62 Result: INSTANCE INSTANCE-ID INSTANCE i-4f8cf126 INSTANCE i-0bb7ca62 24

Discussion r Advantages and disadvantages of using DNS 25

Outline  Admin and recap  Load balance (direction) o Basic load direction mechanisms o DNS o Load balancer (smart switch) 26

Clustering with VIP: Basic Idea r Clients get a single service IP address, called virtual IP address (VIP) r A virtual server (also referred to as load balancer, vserver or smart switch) listens at VIP address and port r A virtual server is bound to a number of physical servers running in a server farm r A client sends a request to the virtual server, which in turn selects a physical server in the server farm and directs this request to the selected physical server 27

Big Picture 28

VIP Clustering server array Clients L4: TCP L7: HTTP SSL etc. Goals server load balancing failure detection access control filtering priorities/QoS request locality transparent caching smart switch virtual IP addresses (VIPs) What to switch/filter on? L3 source IP and/or VIP L4 (TCP) ports etc. L7 URLs and/or cookies L7 SSL session IDs

Load Balancer (LB): Basic Structure 30 LB Client Server1 Server2 Server3 Problem of the basic structure? VIP RIP1 RIP2 RIP3 D=VIPS=client

Problem  Client to server packet has VIP as destination address, but real servers use RIPs o if LB just forwards the packet from client to a real server, the real server may drop the packet o Reply from real server to client has real server IP as source -> client will drop the packet 31 Real Server TCP socket space state: listening address: {*.6789, *:*} completed connection queue: C1; C2 sendbuf: recvbuf: state: established address: { :6789, } sendbuf: recvbuf: state: established address: { :6789, } sendbuf: recvbuf: … … D=VIPS=client

Solution 1: Network Address Translation (NAT)  LB does rewriting/ translation  Thus, the LB is similar to a typical NAT gateway with an additional scheduling function 32 Load Balancer

Example Virtual Server via NAT

LB/NAT Flow 34

LB/NAT Flow 35

SLB/NAT Flow: Details 1. When a user accesses a virtual service provided by the server cluster, a request packet destined for the virtual IP address (the IP address to accept requests for virtual service) arrives at the load balancer. 2. The load balancer examines the packet's destination address and port number. If they match a virtual service in the virtual server rule table, a real server is selected from the cluster by a scheduling algorithm and the connection is added to hash table that records connections. Then, the destination address and the port of the packet are rewritten to those of the selected server, and the packet is forwarded to the server. When an incoming packet belongs to an established connection, the connection can be found in the hash table and the packet is rewritten and forwarded to the right server. 3. The request is processed by one of the physical servers. 4. When response packets come back, the load balancer rewrites the source address and port of the packets to those of the virtual service. When a connection terminates or timeouts, the connection record is removed from the hash table. 5. A reply is sent back to the user. 36

LB/NAT Advantages and Disadvantages  Advantages: o Only one public IP address is needed for the load balancer; real servers can use private IP addresses o Real servers need no change and are not aware of load balancing  Problem o The load balancer must on the critical path o The load balancer may become the bottleneck due to load to rewrite request and response packets Typically, rewriting responses has a lot more load because there are typically a lot more response packets

LB with Direct Reply 38 LB Client Server1 Server2 Server3 Direct reply VIP Each real server uses VIP as its IP address

LB/DR Architecture load balancer Connected by a single switch

Why IP Address Matters?  Each network interface card listens to an assigned MAC address  A router is configured with the range of IP addresses connected to each interface (NIC)  To send to a device with a given IP, the router needs to translate IP to MAC (device) address  The translation is done by the Address Resolution Protocol (ARP) 40 VIP

41 ARP Protocol  ARP is “plug-and-play”: o nodes create their ARP tables without intervention from net administrator  A broadcast protocol: o Router broadcasts query frame, containing queried IP address all machines on LAN receive ARP query o Node with queried IP receives ARP frame, replies its MAC address

ARP in Action 42 VIP - Router broadcasts ARP broadcast query: who has VIP? - ARP reply from LB: I have VIP; my MAC is MAC LB - Data packet from R to LB: destination MAC = MAC LB Router R D=VIPS=client

LB/DR Problem 43 VIP ARP and race condition: When router R gets a packet with dest. address VIP, it broadcasts an Address Resolution Protocol (ARP) request: who has VIP? One of the real servers may reply before load balancer Solution: configure real servers to not respond to ARP request Router R

LB via Direct Routing  The virtual IP address is shared by real servers and the load balancer.  Each real server has a non-ARPing, loopback alias interface configured with the virtual IP address, and the load balancer has an interface configured with the virtual IP address to accept incoming packets.  The workflow of LB/DR is similar to that of LB/NAT: o the load balancer directly routes a packet to the selected server the load balancer simply changes the MAC address of the data frame to that of the server and retransmits it on the LAN (how to know the real server’s MAC?) o When the server receives the forwarded packet, the server determines that the packet is for the address on its loopback alias interface, processes the request, and finally returns the result directly to the user

LB/DR Advantages and Disadvantages  Advantages: o Real servers send response packets to clients directly, avoiding LB as bottleneck  Disadvantages: o Servers must have non-arp alias interface o The load balancer and server must have one of their interfaces in the same LAN segment

Example Implementation of LB  An example open source implementation is Linux virtual server (linux-vs.org) Used by – –sourceforge.net –wikipedia.org More details on ARP problem: HOWTO/HOWTO/LVS-HOWTO.arp_problem.html o Many commercial LB servers from F5, Cisco, …  More details please read chapter 2 of Load Balancing Servers, Firewalls, and CachesLoad Balancing Servers, Firewalls, and Caches 46

Example: Wikipedia Architecture 47

Discussion: Problem of the Load Balancer Architecture 48 LB Client Server1 Server2 Server3 A major remaining problem is that the LB becomes a single point of failure (SPOF). VIP D=VIPS=client

Solutions  Redundant load balancers o E.g., two load balancers  Fully distributed load balancing o e.g., Microsoft Network Load Balancing (NLB) 49

Microsoft NLB  No dedicated load balancer  All servers in the cluster receive all packets  All servers within the cluster simultaneously run a mapping algorithm to determine which server should handle the packet. Those servers not required to service the packet simply discard it.  Mapping (ranking) algorithm: computing the “winning” server according to host priorities, multicast or unicast mode, port rules, affinity, load percentage distribution, client IP address, client port number, other internal load information 50

Discussion  Compare the design of using Load Balancer vs Microsoft NLB 51