1. 2: Application Layer 1 Chapter 2: Application layer r 2.1 Principles of network applications  app architectures  app requirements r 2.2 Web and HTTP r 2.4 Electronic Mail  SMTP, POP3, IMAP r 2.5 DNS r 2.6 P2P applications r 2.7 Socket programming with TCP r 2.8 Socket programming with UDP

2. 2: Application Layer 2 Pure P2P architecture r no always-on server r arbitrary end systems directly communicate r peers are intermittently connected and change IP addresses r Three topics:  File distribution  Searching for information  Case Study: Skype peer-peer

3. 2: Application Layer 3 File Distribution: Server-Client vs P2P Question : How much time to distribute file from one server to N peers? usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) File, size F u s : server upload bandwidth u i : peer i upload bandwidth d i : peer i download bandwidth

4. 2: Application Layer 4 File distribution time: server-client usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) F r server sequentially sends N copies:  NF/u s time r client i takes F/d i time to download increases linearly in N (for large N) = d cs = max { NF/u s, F/min(d i ) } i Time to distribute F to N clients using client/server approach

5. 2: Application Layer 5 File distribution time: P2P usus u2u2 d1d1 d2d2 u1u1 uNuN dNdN Server Network (with abundant bandwidth) F r server must send one copy: F/u s time r client i takes F/d i time to download r NF bits must be downloaded (aggregate) r fastest possible upload rate: u s +  u i d P2P = max { F/u s, F/min(d i ), NF/(u s +  u i ) } i

6. 2: Application Layer 6 Server-client vs. P2P: example Client upload rate = u, F/u = 1 hour, u s = 10u, d min ≥ u s

7. 2: Application Layer 7 File distribution: BitTorrent tracker: tracks peers participating in torrent torrent: group of peers exchanging chunks of a file obtain list of peers trading chunks peer r P2P file distribution

8. 2: Application Layer 8 BitTorrent (1) r file divided into 256KB chunks. r peer joining torrent:  has no chunks, but will accumulate them over time  registers with tracker to get list of peers, connects to subset of peers (“neighbors”) r while downloading, peer uploads chunks to other peers. r peers may come and go r once peer has entire file, it may (selfishly) leave or (altruistically) remain

9. 2: Application Layer 9 BitTorrent (2) Pulling Chunks r at any given time, different peers have different subsets of file chunks r periodically, a peer (Alice) asks each neighbor for list of chunks that they have. r Alice sends requests for her missing chunks  rarest first Sending Chunks: tit-for-tat r Alice sends chunks to four neighbors currently sending her chunks at the highest rate  re-evaluate top 4 every 10 secs r every 30 secs: randomly select another peer, starts sending chunks  newly chosen peer may join top 4  “optimistically unchoke”

10. 2: Application Layer 10 BitTorrent: Tit-for-tat (1) Alice “optimistically unchokes” Bob (2) Alice becomes one of Bob’s top-four providers; Bob reciprocates (3) Bob becomes one of Alice’s top-four providers With higher upload rate, can find better trading partners & get file faster!

11. 2: Application Layer 11 P2P: searching for information File sharing (eg e-mule) r Index dynamically tracks the locations of files that peers share. r Peers need to tell index what they have. r Peers search index to determine where files can be found Instant messaging r Index maps user names to locations. r When user starts IM application, it needs to inform index of its location r Peers search index to determine IP address of user. Index in P2P system: maps information to peer location (location = IP address & port number).

12. 2: Application Layer 12 P2P: centralized index original “Napster” design 1) when peer connects, it informs central server:  IP address  content 2) Alice queries for “Hey Jude” 3) Alice requests file from Bob centralized directory server peers Alice Bob 1 1 1 1 2 3

13. 2: Application Layer 13 P2P: problems with centralized directory r single point of failure r performance bottleneck r copyright infringement: “target” of lawsuit is obvious file transfer is decentralized, but locating content is highly centralized

14. 2: Application Layer 14 Query flooding r fully distributed  no central server r used by Gnutella r Each peer indexes the files it makes available for sharing (and no other files) overlay network: graph r edge between peer X and Y if there’s a TCP connection r all active peers and edges form overlay net r edge: virtual (not physical) link r given peer typically connected with < 10 overlay neighbors

15. 2: Application Layer 15 Query flooding Query QueryHit Query QueryHit Query QueryHit File transfer: HTTP r Query message sent over existing TCP connections r peers forward Query message r QueryHit sent over reverse path Scalability: limited scope flooding

16. 2: Application Layer 16 Gnutella: Peer joining 1. joining peer Alice must find another peer in Gnutella network: use list of candidate peers 2. Alice sequentially attempts TCP connections with candidate peers until connection setup with Bob 3. Flooding: Alice sends Ping message to Bob; Bob forwards Ping message to his overlay neighbors (who then forward to their neighbors….) r peers receiving Ping message respond to Alice with Pong message 4. Alice receives many Pong messages, and can then setup additional TCP connections Peer leaving: see homework problem!

17. Distributed Hash Table (DHT) r DHT: a distributed P2P database r database has (key, value) pairs; examples:  key: ss number; value: human name  key: movie title; value: IP address r Distribute the (key, value) pairs over the (millions of peers) r a peer queries DHT with key  DHT returns values that match the key r peers can also insert (key, value) pairs Application 2-17

18. Q: how to assign keys to peers? r central issue:  assigning (key, value) pairs to peers. r basic idea:  convert each key to an integer  Assign integer to each peer  put (key,value) pair in the peer that is closest to the key Application 2-18

19. DHT identifiers r assign integer identifier to each peer in range [0,2 n -1] for some n.  each identifier represented by n bits. r require each key to be an integer in same range r to get integer key, hash original key  e.g., key = hash(“Led Zeppelin IV”)  this is why its is referred to as a distributed “hash” table Application 2-19

20. Assign keys to peers r rule: assign key to the peer that has the closest ID. r convention in lecture: closest is the immediate successor of the key. r e.g., n=4; peers: 1,3,4,5,8,10,12,14;  key = 13, then successor peer = 14  key = 15, then successor peer = 1 Application 2-20

21. 1 3 4 5 8 10 12 15 Circular DHT (1) r each peer only aware of immediate successor and predecessor. r “overlay network ” Application 2-21

22. 0001 0011 0100 0101 1000 1010 1100 1111 Who’s responsible for key 1110 ? I am O(N) messages on avgerage to resolve query, when there are N peers 1110 Define closest as closest successor Application 2-22 Circular DHT (1)

23. Circular DHT with shortcuts r each peer keeps track of IP addresses of predecessor, successor, short cuts. r reduced from 6 to 2 messages. r possible to design shortcuts so O(log N) neighbors, O(log N) messages in query 1 3 4 5 8 10 12 15 Who’s responsible for key 1110? Application 2-23

24. Peer churn example: peer 5 abruptly leaves r peer 4 detects peer 5 departure; makes 8 its immediate successor; asks 8 who its immediate successor is; makes 8’s immediate successor its second successor. r what if peer 13 wants to join? 1 3 4 5 8 10 12 15 handling peer churn:  peers may come and go (churn)  each peer knows address of its two successors  each peer periodically pings its two successors to check aliveness  if immediate successor leaves, choose next successor as new immediate successor Application 2-24

25. 2: Application Layer 25 Chapter 2: Application layer r 2.1 Principles of network applications r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail  SMTP, POP3, IMAP r 2.5 DNS r 2.6 P2P applications r 2.7 Socket programming with TCP r 2.8 Socket programming with UDP

26. 2: Application Layer 26 Socket programming Socket API r introduced in BSD4.1 UNIX, 1981 r explicitly created, used, released by apps r client/server paradigm r two types of transport service via socket API:  unreliable datagram  reliable, byte stream- oriented a host-local, application-created, OS-controlled interface (a “door”) into which application process can both send and receive messages to/from another application process socket Goal: learn how to build client/server application that communicate using sockets

27. 2: Application Layer 27 Socket-programming using TCP Socket: a door between application process and end- end-transport protocol (UCP or TCP) TCP service: reliable transfer of bytes from one process to another process TCP with buffers, variables socket controlled by application developer controlled by operating system host or server process TCP with buffers, variables socket controlled by application developer controlled by operating system host or server internet

28. 2: Application Layer 28 Socket programming with TCP Client must contact server r server process must first be running r server must have created socket (door) that welcomes client’s contact Client contacts server by: r creating client-local TCP socket r specifying IP address, port number of server process r When client creates socket: client TCP establishes connection to server TCP r When contacted by client, server TCP creates new socket for server process to communicate with client  allows server to talk with multiple clients  source port numbers used to distinguish clients (more in Chap 3) TCP provides reliable, in-order transfer of bytes (“pipe”) between client and server application viewpoint

29. 2: Application Layer 29 Client/server socket interaction: TCP wait for incoming connection request connectionSocket = welcomeSocket.accept() create socket, port= x, for incoming request: welcomeSocket = ServerSocket() create socket, connect to hostid, port= x clientSocket = Socket() close connectionSocket read reply from clientSocket close clientSocket Server (running on hostid ) Client send request using clientSocket read request from connectionSocket write reply to connectionSocket TCP connection setup

30. 2: Application Layer 30 Client process client TCP socket Stream jargon r A stream is a sequence of characters that flow into or out of a process. r An input stream is attached to some input source for the process, e.g., keyboard or socket. r An output stream is attached to an output source, e.g., monitor or socket.

31. 2: Application Layer 31 Socket programming with TCP Example client-server app: 1) client reads line from standard input ( inFromUser stream), sends to server via socket ( outToServer stream) 2) server reads line from socket 3) server converts line to uppercase, sends back to client 4) client reads, prints modified line from socket ( inFromServer stream)

32. 2: Application Layer 32 Example: Java client (TCP) import java.io.*; import java.net.*; class TCPClient { public static void main(String argv[]) throws Exception { String sentence; String modifiedSentence; BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); Socket clientSocket = new Socket("hostname", 6789); DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream()); Create input stream Create client socket, connect to server Create output stream attached to socket

33. 2: Application Layer 33 Example: Java client (TCP), cont. BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream())); sentence = inFromUser.readLine(); outToServer.writeBytes(sentence + '\n'); modifiedSentence = inFromServer.readLine(); System.out.println ("FROM SERVER: " + modifiedSentence ); clientSocket.close(); } Create input stream attached to socket Send line to server Read line from server

34. 2: Application Layer 34 Example: Java server (TCP) import java.io.*; import java.net.*; class TCPServer { public static void main(String argv[]) throws Exception { String clientSentence; String capitalizedSentence; ServerSocket welcomeSocket = new ServerSocket(6789); while(true) { Socket connectionSocket = welcomeSocket.accept(); BufferedReader inFromClient = new BufferedReader(new InputStreamReader(connectionSocket.getInputStream())); Create welcoming socket at port 6789 Wait, on welcoming socket for contact by client Create input stream, attached to socket

35. 2: Application Layer 35 Example: Java server (TCP), cont DataOutputStream outToClient = new DataOutputStream (connectionSocket.getOutputStream()); clientSentence = inFromClient.readLine(); capitalizedSentence = clientSentence.toUpperCase() + '\n'; outToClient.writeBytes(capitalizedSentence); } Read in line from socket Create output stream, attached to socket Write out line to socket End of while loop, loop back and wait for another client connection

36. 2: Application Layer 36 Chapter 2: Application layer r 2.1 Principles of network applications r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail  SMTP, POP3, IMAP r 2.5 DNS r 2.6 P2P applications r 2.7 Socket programming with TCP r 2.8 Socket programming with UDP

37. 2: Application Layer 37 Socket programming with UDP UDP: no “connection” between client and server r no handshaking r sender explicitly attaches IP address and port of destination to each packet r server must extract IP address, port of sender from received packet UDP: transmitted data may be received out of order, or lost application viewpoint UDP provides unreliable transfer of groups of bytes (“datagrams”) between client and server

38. 2: Application Layer 38 Client/server socket interaction: UDP Server (running on hostid ) close clientSocket read datagram from clientSocket create socket, clientSocket = DatagramSocket() Client Create datagram with server IP and port=x; send datagram via clientSocket create socket, port= x. serverSocket = DatagramSocket() read datagram from serverSocket write reply to serverSocket specifying client address, port number

39. 2: Application Layer 39 Example: Java client (UDP) Output: sends packet (recall that TCP sent “byte stream”) Input: receives packet (recall thatTCP received “byte stream”) Client process client UDP socket

40. 2: Application Layer 40 Example: Java client (UDP) import java.io.*; import java.net.*; class UDPClient { public static void main(String args[]) throws Exception { BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in)); DatagramSocket clientSocket = new DatagramSocket(); InetAddress IPAddress = InetAddress.getByName("hostname"); byte[] sendData = new byte[1024]; byte[] receiveData = new byte[1024]; String sentence = inFromUser.readLine(); sendData = sentence.getBytes(); Create input stream Create client socket Translate hostname to IP address using DNS

41. 2: Application Layer 41 Example: Java client (UDP), cont. DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 9876); clientSocket.send(sendPacket); DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); clientSocket.receive(receivePacket); String modifiedSentence = new String(receivePacket.getData()); System.out.println("FROM SERVER:" + modifiedSentence); clientSocket.close(); } Create datagram with data-to-send, length, IP addr, port Send datagram to server Read datagram from server

42. 2: Application Layer 42 Example: Java server (UDP) import java.io.*; import java.net.*; class UDPServer { public static void main(String args[]) throws Exception { DatagramSocket serverSocket = new DatagramSocket(9876); byte[] receiveData = new byte[1024]; byte[] sendData = new byte[1024]; while(true) { DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length); serverSocket.receive(receivePacket); Create datagram socket at port 9876 Create space for received datagram Receive datagram

43. 2: Application Layer 43 Example: Java server (UDP), cont String sentence = new String(receivePacket.getData()); InetAddress IPAddress = receivePacket.getAddress(); int port = receivePacket.getPort(); String capitalizedSentence = sentence.toUpperCase(); sendData = capitalizedSentence.getBytes(); DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, port); serverSocket.send(sendPacket); } Get IP addr port #, of sender Write out datagram to socket End of while loop, loop back and wait for another datagram Create datagram to send to client

44. 2: Application Layer 44 Chapter 2: Summary r application architectures  client-server  P2P  hybrid r application service requirements:  reliability, bandwidth, delay r Internet transport service model  connection-oriented, reliable: TCP  unreliable, datagrams: UDP our study of network apps now complete! r specific protocols:  HTTP  FTP  SMTP, POP, IMAP  DNS  P2P: BitTorrent, Skype r socket programming

45. 2: Application Layer 45 Chapter 2: Summary r typical request/reply message exchange:  client requests info or service  server responds with data, status code r message formats:  headers: fields giving info about data  data: info being communicated Most importantly: learned about protocols Important themes: r control vs. data msgs  in-band, out-of-band r centralized vs. decentralized r stateless vs. stateful r reliable vs. unreliable msg transfer r “complexity at network edge”