1. Networking with Java 1

2. Introduction to Networking 2

3. Protocols 3 Hi TCP connection request Hi TCP connection reply Got the time? GET http://www.google.com 2:00 time

4. Internet Architecture Model 4 Transport (TCP,UDP) DATA Link (LINK) HEADER DATA Network (IP) HEADERDATA Application (HTTP, FTP) DATA HEADER

5. TCP ( Transmission-Control Protocol) Enables symmetric byte-stream transmission between two endpoints (applications) Reliable communication channel TCP perform these tasks: – connection establishment by handshake (relatively slow) – division to numbered packets (transferred by IP) – error correction of packets (checksum) – acknowledgement and retransmission of packets – connection termination by handshake 5

6. UDP (User Datagram Protocol) Enables direct datagram (packet) transmission from one endpoint to another No reliability (except for data correction) – sender does not wait for acknowledgements – arrival order is not guaranteed – arrival is not guaranteed Used when speed is essential, even in cost of reliability – e.g., streaming media, games, Internet telephony, etc. 6

7. Ports A computer may have several applications that communicate with applications on remote computers through the same physical connection to the network When receiving a packet, how can the computer tell which application is the destination? Solution: each channel endpoint is assigned a unique port that is known to both the computer and the other endpoint 7

8. Ports (cont) Thus, an endpoint application on the Internet is identified by – A host name 32 bits IP-address – A 16 bits port Why dont we specify the port in a Web browser? 8

9. Known Ports Some known ports are – 20, 21: FTP – 23: TELNET – 25: SMTP – 110: POP3 – 80: HTTP – 119: NNTP 9 21 23 25 110 80 119 Client Application web browser mail client

10. Sockets A socket is a construct that represents one end-point of a two-way communication channel between two programs running on the network Using sockets, the OS provides processes a file-like access to the channel – i.e., sockets are allocated a file descriptor, and processes can access (read/write) the socket by specifying that descriptor A specific socket is identified by the machine's IP and a port within that machine 10

11. Sockets (cont) A socket stores the IP and port number of the other end-point computer of the channel When writing to a socket, the written bytes are sent to the other computer and port (e.g., over TCP/IP) – That is, remote IP and port are attached to the packets When OS receives packets on the network, it uses their destination port to decide which socket should get the received bytes 11

12. Java Sockets Low-Level Networking 12

13. Java Sockets Java wraps OS sockets (over TCP) by the objects of class java.net.Socket new Socket(String remoteHost, int remotePort) creates a TCP socket and connects it to the remote host on the remote port (hand shake) Write and read using streams: – InputStream getInputStream() – OutputStream getOutputStream() 13

14. A Socket Example 14 import java.net.*; import java.io.*; public class SimpleSocket { public static void main(String[] args) throws IOException {... next slide... }

15. 15 Socket socket = new Socket("www.google.com", 80); InputStream istream = socket.getInputStream(); OutputStream ostream = socket.getOutputStream(); String request = "GET index.html HTTP/1.1\r\n" + "Host: www.cs.biu.ac.il\r\n" + "Connection: close\r\n\r\n"; ostream.write(request.getBytes()); byte[] response = new byte[4096]; int bytesRead = -1; while ((bytesRead = istream.read(response)) >= 0) { System.out.write(response, 0, bytesRead); } socket.close(); Needed for forwarding for example

16. Timeout You can set timeout values to blocking method read() of Socket Use the method socket. setSoTimeout(milliseconds) If timeout is reached before the method returns, java.net.SocketTimeoutException is thrown 16

17. Java Sockets and HTTP 17

18. HTTP Message Structure A HTTP message has the following structure: 18 Request/Status-Line \r\n Header1: value1 \r\n Header2: value2 \r\n... HeaderN: valueN \r\n \r\n Message-Body

19. Reading HTTP Messages Several ways to interpret the bytes of the body – Binary: images, compressed files, class files,... – Text: ASCII, Latin-1, UTF-8,... Commonly, applications parse the headers of the message, and process the body according to the information supplied by the headers – E.g., Content-Type, Content-Encoding, Transfer- Encoding 19

20. An Example 20

21. Parsing the Headers So how are the headers themselves are represented? The headers of a HTTP message must be in US-ASCII format (1 byte per character) 21

22. Example: Extracting the Headers 22 Socket socket = new Socket(argv[0], 80); InputStream istream = socket.getInputStream(); OutputStream ostream = socket.getOutputStream(); String request = "GET / HTTP/1.1\r\n" + "Host: " + argv[0] + "\r\n" + "Connection: close\r\n\r\n"; ostream.write(request.getBytes()); StringBuffer headers = new StringBuffer(); int byteRead = 0; while ( !endOfHeaders(headers) && (byteRead = istream.read()) >= 0) { headers.append((char) byteRead); } System.out.print(headers); socket.close();

23. Example: Extracting the Headers (cont) 23 public static boolean endOfHeaders(StringBuffer headers) { int lastIndex = headers.length() - 1; if (lastIndex < 3 || headers.charAt(lastIndex) != '\n') return false; return ( headers.substring(lastIndex - 3, lastIndex + 1).equals("\r\n\r\n")); }

24. Persistent Connections According to HTTP/1.1, a server does not have to close the connection after fulfilling your request One connection (socket) can be used for several requests and responses send more requests – even while earlier responses are being transferred (pipelining) – saves slow start time how can the client know when one response ends and a new one begins? To avoid persistency, require explicitly by the header: Connection: close 24

25. URL and URLConnection High-Level Networking 25

26. Working with URLs URL (Uniform/Universal Resource Locator): a reference (an address) to a resource on the Internet http://www.cs.huji.ac.il:80/~dbi/main.html#notes 26 Protocol Host Name Port Number File Name Reference

27. The Class URL The class URL is used for parsing URLs Constructing URLs: –URL w3c1 = new URL("http://www.w3.org/TR/"); –URL w3c2 = new URL("http","www.w3.org",80,"TR/"); –URL w3c3 = new URL(w3c2, "xhtml1/"); If the string is not an absolute URL, then it is considered relative to the URL 27

28. Parsing URLs The following methods of URL can be used for parsing URLs getProtocol(), getHost(), getPort(), getPath(), getFile(), getQuery(), getRef() 28

29. The class URLConnection To establish the actual resource, we can use the object URLConnection obtained by url.openConnection() If the protocol of the URL is HTTP, the returned object is of class HttpURLConnection This class encapsulates all socket management and HTTP directions required to obtain the resource 29

30. 30 public class ContentExtractor { public static void main(String[] argv) throws Exception { URL url = new URL(argv[0]); System.out.println("Host: " + url.getHost()); System.out.println("Protocol: " + url.getProtocol()); System.out.println("----"); URLConnection con = url.openConnection(); InputStream stream = con.getInputStream(); byte[] data = new byte[4096]; int bytesRead = 0; while((bytesRead=stream.read(data))>=0) { System.out.write(data,0,bytesRead); }}}

31. About URLConnection The life cycle of a URLConnection object has two parts: – Before actual connection establishment Connection configuration – After actual connection establishment Content retrieval Passage from the first phase to the second is implicit – A result of calling some committing methods, like getDate() 31

32. About HttpURLConnection The HttpURLConnection class encapsulates all HTTP transaction over sockets, e.g., – Content decoding – Redirection – Proxy indirection You can control requests by its methods –setRequestMethod, setFollowRedirects, setRequestProperty,... 32 Read more about HttpURLConnection ClassHttpURLConnection Class

33. URLEncoder Contains a utility method encode for converting a string into an encoded format (used in URLs) To convert a string, each character is examined in turn: – Space is converted into a plus sign + –a - z, A - Z, 0 - 9,., -, * and _ remain the same. – The bytes of all special characters are replaced by hexadecimal numbers, preceded with % To decode an encoded string, use decode() of the class URLDecoder 33

34. Client-Server Model A common paradigm for distributed applications Asymmetry in connection establishment: – Server waits for client requests (daemon) at a well known address (IP+port) – Connection is established upon client request Once the connection is made, it can be either symmetric (TELNET) or asymmetric (HTTP) For example: Web servers and browsers 34

35. Client-Server Interaction 35

36. Client-Server Interaction 36

37. Client-Server Interaction 37

38. Client-Server Interaction 38

39. Client-Server Interaction 39

40. Java Sever Sockets 40

41. Java Sockets – A Reminder Java wraps OS sockets (over TCP) by the objects of class java.net.Socket new Socket(String remoteHost, int remotePort) creates a TCP socket and connects it to the remote host on the remote port (hand shake) Write and read using streams: – InputStream getInputStream() – OutputStream getOutputStream() 41

42. Java ServerSocket ServerSocket represents a socket that listens and waits for requests from clients Construction: –new ServerSocket(int port) – Why do we want to specify the port? Listen and accept incoming connections –Socket accept() – returns a new socket (with a new port) for the new channel – blocks until connection is made 42

43. 43 public class EchoServer { public static void main(String[] args) throws IOException { ServerSocket serverSocket = new ServerSocket(8000); Socket socket = null; while (true) { try {... next slide... } catch (IOException exp) {... } finally { try {if (!socket.isClosed()) socket.close(); } catch (IOException e) {} } }}}

44. 44 socket = serverSocket.accept(); String clientName = socket.getInetAddress().getHostName(); BufferedReader reader = new BufferedReader( new InputStreamReader(socket.getInputStream())); PrintStream writer = new PrintStream(socket.getOutputStream()); writer.println("Hello " + clientName + "!"); writer.flush(); String lineRead = null; while ((lineRead = reader.readLine()) != null) { writer.println("You wrote: " + lineRead); writer.flush(); } bytes chars

45. Accepting Connections Usually, the accept() method is executed within an infinite loop – i.e., while(true){...} Whenever accept() returns, a new thread is launched to handle that interaction (not in our example) Hence, the server can handle several requests concurrently 45

46. Timeout You can set timeout values to the blocking method –accept() of ServerSocket Use the method serverSocket. setSoTimeout(milliseconds) If timeout is reached before the method returns, java.net.SocketTimeoutException is thrown 46

47. Get Vs. Post Get – Returns the content of the requested URL – Usually a static resource Post – A block of data is sent with the request – Usually a program or a form 47

48. ContentExtractor – A Reminder 48 public class ContentExtractor { public static void main(String[] argv) throws Exception { URL url = new URL(argv[0]); System.out.println("Host: " + url.getHost()); System.out.println("Protocol: " + url.getProtocol()); System.out.println("----"); HttpURLConnection con = url.openConnection(); InputStream stream = con.getInputStream(); byte[] data = new byte[4096]; int bytesRead = 0; while((bytesRead=stream.read(data))>=0) { System.out.write(data,0,bytesRead); }}}

49. Sending POST Requests In order to send POST requests with HttpURLConnection, you have to do the following: – Enable connection output: con.setDoOutput(true); – Get the output stream: con.getOutputStream() This changes the method from GET to POST – Write the message body into the output stream – close the output stream (important!) 49

50. POST Example - SearchWalla 50 URL url = new URL("http://find.walla.co.il/"); URLConnection connection = url.openConnection(); connection.setDoOutput(true); String query = "q=" + URLEncoder.encode(argv[0], "UTF-8"); OutputStream out = connection.getOutputStream(); out.write(query.getBytes()); out.close(); InputStream in = connection.getInputStream(); int bytesRead = -1; byte[] response = new byte[4096]; while ((bytesRead = in.read(response)) >= 0) System.out.write(response, 0, bytesRead); in.close();

51. Sent Headers 51 POST / HTTP/1.1 User-Agent: Java/1.5.0_01 Host: find.walla.co.il Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive Content-type: application/x-www-form-urlencoded Content-Length: 18 q=Java+networking java SearchWalla "Java networking"

52. Not Covered: UDP Connections The URL, URLConnection, Socket and ServerSocket classes all use the TCP protocol to communicate over the network Java programs can also use UDP protocol for communication For that, use DatagramPacket, DatagramSocket, and MulticastSocket classes 52