1. CMPE 252A : Computer Networks
Chen Qian
Computer Science and Engineering
UCSC Baskin Engineering
Data link layer

2. MAC addresses and ARP 32-bit IP address:
network-layer address for interface
used for layer 3 (network layer) forwarding
MAC (or LAN or physical or Ethernet) address:
function: used ‘locally” to get frame from one interface to another physically-connected interface (same network, in IP-addressing sense)
48 bit MAC address (for most LANs) burned in NIC ROM, also sometimes software settable
e.g.: 1A-2F-BB AD
hexadecimal (base 16) notation
(each “number” represents 4 bits)
Link Layer

3. LAN addresses and ARP each adapter on LAN has unique LAN address LAN
1A-2F-BB AD
LAN
(wired or
wireless)
adapter
71-65-F7-2B-08-53
58-23-D7-FA-20-B0
0C-C4-11-6F-E3-98
Link Layer

4. LAN addresses (more) MAC address allocation administered by IEEE
manufacturer buys portion of MAC address space (to assure uniqueness)
analogy:
MAC address: like Social Security Number
IP address: like postal address
MAC flat address ➜ portability
can move LAN card from one LAN to another
IP hierarchical address not portable
address depends on IP subnet to which node is attached
Link Layer

5. ARP: address resolution protocol
Question: how to determine
interface’s MAC address, knowing its IP address?
ARP table: each IP node (host, router) on LAN has table
IP/MAC address mappings for some LAN nodes:
< IP address; MAC address; TTL>
TTL (Time To Live): time after which address mapping will be forgotten (typically 20 min)
1A-2F-BB AD
LAN
71-65-F7-2B-08-53
58-23-D7-FA-20-B0
0C-C4-11-6F-E3-98
Link Layer

6. ARP protocol: same LAN A wants to send datagram to B
B’s MAC address not in A’s ARP table.
A broadcasts ARP query packet, containing B's IP address
dest MAC address = FF-FF-FF-FF-FF-FF
all nodes on LAN receive ARP query
B receives ARP packet, replies to A with its (B's) MAC address
frame sent to A’s MAC address (unicast)
A caches (saves) IP-to-MAC address pair in its ARP table until information becomes old (times out)
soft state: information that times out (goes away) unless refreshed
Link Layer

7. Addressing: routing to another LAN
walkthrough: send datagram from A to B via R
focus on addressing – at IP (datagram) and MAC layer (frame)
assume A knows B’s IP address
assume A knows IP address of first hop router, R (how?)
DHCP
assume A knows R’s MAC address (how?)
ARP R
1A-23-F9-CD-06-9B
E6-E BB-4B
CC-49-DE-D0-AB-7D
C-E8-FF-55 A
49-BD-D2-C7-56-2A
88-B2-2F-54-1A-0F B
Link Layer

8. Addressing: routing to another LAN
A creates IP datagram with IP source A, destination B
A creates link-layer frame with R's MAC address as dest, frame contains A-to-B IP datagram
MAC src: C-E8-FF-55
MAC dest: E6-E BB-4B IP
Eth
Phy
IP src:
IP dest: R
1A-23-F9-CD-06-9B
E6-E BB-4B
CC-49-DE-D0-AB-7D
C-E8-FF-55 A
49-BD-D2-C7-56-2A
88-B2-2F-54-1A-0F B
Link Layer

9. Addressing: routing to another LAN
frame sent from A to R
frame received at R, datagram removed, passed up to IP
MAC src: C-E8-FF-55
MAC dest: E6-E BB-4B
IP src:
IP dest:
IP src:
IP dest: IP
Eth
Phy IP
Eth
Phy R
1A-23-F9-CD-06-9B
E6-E BB-4B
CC-49-DE-D0-AB-7D
C-E8-FF-55 A
49-BD-D2-C7-56-2A
88-B2-2F-54-1A-0F B
Link Layer

10. Addressing: routing to another LAN
R forwards datagram with IP source A, destination B
R creates link-layer frame with B's MAC address as dest, frame contains A-to-B IP datagram
MAC src: 1A-23-F9-CD-06-9B
MAC dest: 49-BD-D2-C7-56-2A IP
Eth
Phy
IP src:
IP dest: IP
Eth
Phy R
1A-23-F9-CD-06-9B
E6-E BB-4B
CC-49-DE-D0-AB-7D
C-E8-FF-55 A
49-BD-D2-C7-56-2A
88-B2-2F-54-1A-0F B
Link Layer

11. Addressing: routing to another LAN
R forwards datagram with IP source A, destination B
R creates link-layer frame with B's MAC address as dest, frame contains A-to-B IP datagram
IP src:
IP dest:
MAC src: 1A-23-F9-CD-06-9B
MAC dest: 49-BD-D2-C7-56-2A IP
Eth
Phy IP
Eth
Phy R
1A-23-F9-CD-06-9B
E6-E BB-4B
CC-49-DE-D0-AB-7D
C-E8-FF-55 A
49-BD-D2-C7-56-2A
88-B2-2F-54-1A-0F B
Link Layer

12. Addressing: routing to another LAN
R forwards datagram with IP source A, destination B
R creates link-layer frame with B's MAC address as dest, frame contains A-to-B IP datagram
MAC src: 1A-23-F9-CD-06-9B
MAC dest: 49-BD-D2-C7-56-2A
IP src:
IP dest: IP
Eth
Phy B A R
49-BD-D2-C7-56-2A
C-E8-FF-55
1A-23-F9-CD-06-9B
E6-E BB-4B
CC-49-DE-D0-AB-7D
88-B2-2F-54-1A-0F
Link Layer

13. Ethernet switch link-layer device: takes an active role
store, forward Ethernet frames
examine incoming frame’s MAC address, selectively forward frame to one-or-more outgoing links when frame is to be forwarded on segment, uses CSMA/CD to access segment
transparent
hosts are unaware of presence of switches
plug-and-play, self-learning
switches do not need to be configured
Link Layer

14. Switch: multiple simultaneous transmissions
hosts have dedicated, direct connection to switch
switches buffer packets
Ethernet protocol used on each incoming link, but no collisions; full duplex
each link is its own collision domain
switching: A-to-A’ and B-to-B’ can transmit simultaneously, without collisions
switch with six interfaces
(1,2,3,4,5,6) A A’ B B’ C C’ 1 2 3 4 5 6
Link Layer

15. Switch forwarding table
Q: how does switch know A’ reachable via interface 4, B’ reachable via interface 5?
switch with six interfaces
(1,2,3,4,5,6) A A’ B B’ C C’ 1 2 3 4 5 6
A: each switch has a switch table, each entry:
(MAC address of host, interface to reach host, time stamp)
looks like a routing table!
Q: how are entries created, maintained in switch table?
something like a routing protocol?
Link Layer

16. Switch: self-learning
Source: A
Dest: A’ A A’ B B’ C C’ 1 2 3 4 5 6
A A’
switch learns which hosts can be reached through which interfaces
when frame received, switch “learns” location of sender: incoming LAN segment
records sender/location pair in switch table
MAC addr interface TTL A 1 60
Switch table
(initially empty)
Link Layer

17. Self-learning, forwarding: example
Source: A
Dest: A’ A A’ B B’ C C’ 1 2 3 4 5 6
A A’
frame destination, A’, locaton unknown:
flood
destination A location known:
selectively send
on just one link
A A’
A A’
A A’
A A’
A A’
A’ A
MAC addr interface TTL A 1 60
switch table
(initially empty) A’ 4 60
Link Layer

18. Interconnecting switches
switches can be connected together D E F S2 S4 S3 H I G A B S1 C
Q: sending from A to G - how does S1 know to forward frame destined to F via S4 and S3?
A: self learning! (works exactly the same as in single-switch case!)
Link Layer

19. Institutional network
mail server
to external
network
web server
router
IP subnet
Link Layer

20. They become more and more similar!
Switches vs. routers
application
transport
network
link
physical
both are store-and-forward:
routers: network-layer devices (examine network-layer headers)
switches: link-layer devices (examine link-layer headers)
both have forwarding tables:
routers: compute tables using routing algorithms, IP addresses
switches: learn forwarding table using flooding, learning, MAC addresses
datagram
frame
link
physical
frame
switch
network
link
physical
datagram
frame
application
transport
network
link
physical
They become more and more similar!
Link Layer

21. Data center networks
10’s to 100’s of thousands of hosts, often closely coupled, in close proximity:
e-business (e.g. Amazon)
content-servers (e.g., YouTube, Akamai, Apple, Microsoft)
search engines, data mining (e.g., Google)
challenges:
multiple applications, each serving massive numbers of clients
managing/balancing load, avoiding processing, networking, data bottlenecks
Inside a 40-ft Microsoft container,
Chicago data center
Link Layer

22. Data center networks load balancer: application-layer routing Internet
receives external client requests
directs workload within data center
returns results to external client (hiding data center internals from client)
Internet
Border router
Load
balancer
Load
balancer
Access router
Tier-1 switches B A C
Tier-2 switches
TOR switches
Server racks 1 2 3 4 5 6 7 8
Link Layer

23. A Scalable, Commodity Data Center Network Architecture
Mohammad Al-Fares
Alexander Loukissas
Amin Vahdat

24. Oversubscription Ratio
……………
Server 2
Server n B
Upper Link Bandwidth(UB) B
Server 1
Oversubscription Ratio= B*n/UB

25. Current Data Center Topology
Edge hosts connect to 1G Top of Rack (ToR) switch
ToR switches connect to 10G
End of Row (EoR) switches
Large clusters: EoR switches
to 10G core switches
Oversubscription of 2.5:1
to 8:1 typical in guidelines
No story for what happens as we move to 10G to the edge
Key challenges: performance, cost, routing, energy, cabling

26. Scalable interconnection bandwidth
Design Goals
Scalable interconnection bandwidth
Arbitrary host communication at full bandwidth
Economies of scale
Commodity Switch
Backward compatibility
Compatible with hosts running Ethernet and IP

27. k/2 Aggregation Switch in each pod
Fat-Tree Topology
K Pods
k/2 Aggregation Switch in each pod
k/2 Edge Switches in each pod
k/2 servers in each Rack

28. Fat-tree Topology Equivalent

29. Routing (k/2)*(k/2) shortest path! IP needs extension here!
Single-Path Routing VS Multi-Path Routing
Static VS Dynamic

30. ECMP（Equal-Cost Multiple-Path Routing)
Static Flow scheduling
limited multiplicity of path to 8-16
Advantage: No packet reordering! Modern Switch support!
Extract Source and Destination Address
Hash Function(CRC16)
Determine which region fall in 1 2 3 4
Hash-Threshold

31. Two-level Routing Table
/24
Routing Aggregation
/24
/24
/24 1
/24
/24
/24 1
/24
/24

32. Two-level Routing Table
Addressing
Using /8 private IP address
Pod Switch: 10. pod. Switch.1.
pod range is [0, k-1](left to right)
switch range is [0, k-1] (left to right, bottom to top)
Core Switch: 10. k. i . j
(i,j) is the point in (k/2)*(k/2) grid
Host: 10.pod. Switch.ID
ID range is [2, k/2+1] (left to right)

33. Two-level Routing Table
Two-level Routing Table

34. Two-level Routing Table
Two-level Routing Table Structure
Two-level Routing Table implementation
TCAM=Ternary Content-Addressable Memory
Parallel searching
Priority encoding

35. Two-level Routing Table---example
Prefix
Outgoing Port
/24
/24 1 /0
Suffix
Outgoing Port /8 3 /8 2
Prefix
Outgoing Port
/16
/16 1
/16 2
/16 3
Two-level Routing Table---example
Prefix
Outgoing Port
/24
/24 1 /0
example
Prefix
Outgoing Port
/32
/32 1 /0
Suffix
Outgoing Port /8 3 /8 2
Suffix
Outgoing Port /8 2 /8 3
Prefix
Outgoing Port
/32
/32 1 /0
Suffix
Outgoing Port /8 2 /8 3

36. Two-Level Routing Table
Avoid Packet Reordering
traffic diffusion occurs in the first half of a packet journey
Centralized Protocol to Initialize the Routing Table

37. Flow Classification (Dynamic)
Soft State (Compatible with Two-Level Routing Table)
A flow=packet with the same source and destination IP address
Avoid Reordering of Flow
Balancing
Assignment and Updating

38. Flow Classification—Flow Assignment
Hash(Src,Des)
Have seen this hash value?
Lookup previously assign port x
Send packet on port x Y
Record new flow record f
Assign f to least-loaded port x N

39. Flow Classification—Update

40. Flow Scheduling
distribution of transfer times and burst lengths of Internet traffic is long-tailed
Large flow dominating
Large flow should be specially handled
Path-level scheduling – will be discussed next class in the paper Hedera

41. Power and Heat

42. Experiment Description—hierarchical tree,click
four machines running four hosts each, and four machines each running four pod switches with one additional uplink
The four pod switches are connected to a 4- port core switch running on a dedicated machine.
3.6:1 oversubscription on the uplinks from the pod switches to the core switch
Each host generates a constant 96Mbit/s of outgoing traffic

43. Result

44. Conclusion
Bandwidth is the scalability bottleneck in large scale clusters
Existing solutions are expensive and limit cluster size
Fat-tree topology with scalable routing and backward compatibility with TCP/IP and Ethernet