Data Center Network Jin Kim GSDC Data Grid Computing School1

2

Agenda Network Basic Knowledge Data Center Network GSDC Data Grid Computing School3

Network History – 1960`s : transmit bits across a communication medium – 1970`s : transmit packets across a communication medium – 1980`s : provide communication services across a series of interconnected network – 1990`s : provide high-speed, broadband communication services to support high- performance computing and multimedia applications across the globe? – 2000`s~ : more, more GSDC Data Grid Computing School4

Internet History – 1969 : ARPANET – 1971 : ARPANET (23 hosts) – 1973 : ARPANET (England-Norway) – 1982 : term “Internet” – 1992 : Internet Society is chartered. World Wide Web released by CERN GSDC Data Grid Computing School5

Common communication tasks Data encoding Signal generation Synchronization Error detection and correction Flow control Multiplexing Addressing Routing Message formatting Security System mgmt GSDC Data Grid Computing School6

Types of communication networks Local Area Network (LAN) Metropolitan Area Network (MAN) Wide Area Network (WAN) GSDC Data Grid Computing School7

ISO-OSI reference model International Standards Organization – Open Systems Interconnection reference model is a framework for connecting computers on a network Motivation – Reduce the complexity of networking software – Support various protocols GSDC Data Grid Computing School8

OSI 7 Layer The McGraw-Hill Companies, Inc., 2000 GSDC Data Grid Computing School9

OSI 7 Layer(2/3) The McGraw-Hill Companies, Inc., 2000 DATA Segment Packet Frame Bit Data Unit GSDC Data Grid Computing School10

OSI 7 Layer(3/3) The McGraw-Hill Companies, Inc., 2000 Port Addr IP Addr MAC Addr GSDC Data Grid Computing School11

Networking layers GSDC Data Grid Computing School Application OS Controller Physical The Anarchistic model 12

Based on TCP/IP TCP/IP is – A suite of protocols – Rules for sending and receiving data across networks – Addressing – Mgmt and verification GSDC Data Grid Computing School13

DNS (Domain Name System) GSDC Data Grid Computing School14 string.or.co string national normal

Basic communication step GSDC Data Grid Computing School15 A`s pkt B`s mac? Arp broadcast A`s pkt send to B SYN pkt to B B`s SYN+ACK? ACK pkt to B Physical layer complete TCP connection establish (3-way handshaking) TCP connection termination (4-way handshaking) TCP SYN_Flooding attack B wait ACK pkt by A for 75 sec

IP Resolved IP – –... 0~2 / ~255 – 192….. IPv4 / IPv6 Allocation IP GSDC Data Grid Computing School16

Else.. Wifi vs 3,4,5 G – Internet connection for host (PC, mobile) – Mobility? GSDC Data Grid Computing School LG

Date Center Network GSDC Data Grid Computing School18

GSDC Data Grid Computing School19

Data center Data center is a pool of resources(computational, storage, network) interconnected using a communication network GSDC Data Grid Computing School20

Datacenter Type – Three-tier – Fat tree: High throughput, low latency (supercomputing interconnecting) – Dcell (multiple NIC to connect host directly) Structure – Tree: several depth (north-south traffic) – Spin-leaf: 2 depth only (east-west traffic) Performance factor – Latency, throughput -> traffic pattern Core Aggregation Access GSDC Data Grid Computing School21

Datacenter Unstructured cabling Structured cabling GSDC Data Grid Computing School22

Cables GSDC Data Grid Computing School23 10G SPF+40/100G QSPF+120G CXP

Datacenter (SW location) GSDC Data Grid Computing School24

Datacenter Schematic of Facebook data center fabric network topology GSDC Data Grid Computing School25

Characteristic Multi layer: latency, complexity, QoS, virtualization East-west vs north-south System add: case-by-case GSDC Data Grid Computing School26

Traffic pattern North-south traffic - internet connecting - SW-SW, SW-Router GSDC Data Grid Computing School East-west traffic (computing with storage connection) - Host-Host, Host-Storage 27

GSDC DCN GSDC Data Grid Computing School28

Main rack switch 설치 전 (Public, Private)Main rack switch 설치 후 (External, Internal) Cable hell GSDC Data Grid Computing School29

Force10 Z9000(External Main Switch) R00E01 (VLT Primary) R00E02 (VLT Secondary) 패치패널 #1 패치패널 #2 패치패널 #3 10G Switch and Patch panel GSDC Data Grid Computing School30

GSDC Data Grid Computing School GSDCN(LAN) 31

KISTI 시카고 대전 #1 OME6500 대전 #2 OME6500 시카고 MX960 GSDC 네트워크 시카고 OME6500 대전 MX960 연구망 GSDCN(WAN) Lightening Signal Synch issue GSDC Data Grid Computing School32

FAST NETWORK? GSDC Data Grid Computing School33

GSDC/KIS 대전 GSDC/KIS 대전 msterdam SKB(EAC+PC-1) SKB(EAC+UNITY) SURFnet (CANARI E) SURFnet (SURFnet) SKB(Level3) Seattle LA New York GSDC/KISTI – CERN 10G LHCOPN SKB SURFnet SURFnet (SURFnet) GSDC Data Grid Computing School34

KREONET GSDC Data Grid Computing School35

GLORIAD GSDC Data Grid Computing School36

TEIN GSDC Data Grid Computing School37

Wan connection GSDC Data Grid Computing School38

Submarine cable GSDC Data Grid Computing School39

LAN and WAN GSDC Data Grid Computing School40

LHCOPN data transfer 가속기 데이터 전용 10G 네트워크 개통을 위한 테스트 Iperf/iperf3/OWPING 10G 네트워크 성능 측정 을 위한 테스트 GSDC Data Grid Computing School41

Q & A GSDC Data Grid Computing School42

IPv4 header GSDC Data Grid Computing School43

TCP vs UDP header GSDC Data Grid Computing School44

LLDP and IPv6 Header GSDC Data Grid Computing School45