1. Datacenter Interconnection Network Design
Christina Delimitrou, Frank Nothaft,
Milad Mohammadi, Laura Sharpless
May 26th 2011 – Final Project Presentation

2. Introduction
Objective: Optimize for cost and power in data-center networks with competitive performance
Evaluate alternative configurations in terms of Performance/Watt, Performance/$.
Solution:
Dragonfly topology
PAR routing
Virtual cut-through flow control

3. Outline
Topology
Cost
Routing / Slicing
Traffic Management
Results

4. Topology

5. Dragonfly
Fat Tree: Common interconnection network in large-scale datacenters
Dragonfly: 
Cost effective use of resources
Fewer hops (improves latency)
Greater path diversity
Fewer optical cables (improves cost)

6. Design Overview
19 Columns
19 Columns

7. Design Overview 19 Columns 19 Columns 9 racks=1/2Group

8. Design Overview Routers per Group = 8 R R R R 1m 3m R R R R
9 racks = 1/2Group
SHARED
0.25m R R R R 1m 3m
0.25m
SHARED R R R R
1 Group = 18 racks
Routers per Group = 8

9. Design Overview
Router #0
Router #1
Router #2
Router #3

10. Design Overview 19 Columns 19 Columns
9 racks=1/2Group
1 Group
Cost Optimization: Connect neighboring
groups with electrical wires
19 Columns

11. Router Pins 7

12. Cost

13. Cost Free variables: number of groups, endpoints / router
Routers / group, endpoints / router => number of groups
Number of routers / group, endpoints / router determines optical cables needed

14. Cost

15. Cost Design Point: 8 routers / group 71 endpoints / router
Group size allows for connecting all routers within a group with unrepeated electrical cables
Connecting to neighboring groups with electrical cables instead of optical
Didn't want lowest cost design point since that will limit potential throughput

16. Cost

17. Energy

18. Latency

19. Routing / Slicing

20. Progressive Adaptive Routing
Adaptive routing: handles tree saturation

21. PAR Implementation: 4 VC's to avoid deadlock
VC0: Min routing in src group
VC1: Val routing to intermediate
VC2: Min routing to dest group
VC3: Min routing within dest group
Next: Threshold determination based on simulation:
                30 + (H x q)non-minimal > (H x q)minimal

22. Slicing Cloud Node Cloud Node We simulate: Two Full Groups
One Cloud Node
Cloud Node
Cloud Node
We started slicing in a simpler way and then decided to simplify the slice
The original thought was to design one group and routers of each group and have a cloud.
1) latency of non-minimal routing
2) latency of minimal routing to the cloud
3) frequency of 
3) number of non-minimal routing
3)  late
Group 0
Group 1

23. Traffic Management

24. Traffic Management Challenges: Cloud abstraction
Simulate PAR misrouting
Cloud abstraction:
1136 nodes = ~1% of total
Assumption: cloud node issues one packet per cycle
number of cloud nodes * ratio of cloud <-> real
Generation of packets leaving the cloud

25. Traffic Management Traffic Misrouting
Misrouting due to PAR varies inversely with offered traffic
Range is linear and fairly small (varies from 15-20%)
Currently, we say 20% of all packets are misrouted, and  1% of packets sent between nodes in the cloud are misrouted to the outside world
Hotspot Placement
Randomly place hotspots in topology
Guarantee at least one non-cloud hotspot 

26. Results

27. Throughput
Simulation Cycles

28. Latency
Simulation Cycles

29. Simulator Status Topology supports slicing Router supports PAR
Traffic Manager Supports Hot Spot and Slicing
Partly implemented message routing

30. Next? Reevaluate PAR implementation Complete Hot Spot implementation
Complete message routing implementation

31. Thank you!