1. COMP201 Java Programming Part III: Advanced Features Topic 13: Networking Volume II,Chapter 3

2. COMP201 Topic 14 / Slide 2 Objective and Outline l Objective: n Introduction to Java networking features –It is much easier to write networking programs in Java than in C++ –But less efficient. l Outline n Networking basics –IP addresses, ports, protocols, client-server interaction n Socket-level programming –Writing a client (Socket) –Writing a server (ServerSocket) –Example: writing your own icq n Communicating with web servers –Retrieving information (URL, URLConnection) –Sending information

3. COMP201 Topic 14 / Slide 3 Networking Basics l Internet protocol (IP) addresses n Every host on Internet has a unique IP address 143.89.40.46, 203.184.197.198 203.184.197.196, 203.184.197.197, 127.0.0.1 n More convenient to refer to using hostname string cs.ust.hk, tom.com, localhost n One hostname can correspond to multiple internet addresses: –www.yahoo.com:www.yahoo.com 66.218.70.49; 66.218.70.50; 66.218.71.80; 66.218.71.84; … n Domain Naming Service (DNS) maps names to numbers

4. COMP201 Topic 14 / Slide 4 java.net.InetAddress class converts between hostnames and internet addresses InetAddress tm= InetAddress.getByName(“www.yahoo.com"); InetAddress tm= InetAddress.getByName(“localhost"); //127.0.0.1 InetAddress tm = InetAddress.getLocalHost(); l Can get array of addresses (if more than one) InetAddress[] addrs; addrs=InetAddress.getAllByName(“www.yahoo.com"); for (int i = 0; i < addr.length; i++) System.out.println(addrs[i].getHostAddress()); InetAddressTest.java Networking Basics

5. COMP201 Topic 14 / Slide 5 l Ports l Many different services can be running on the host l A port identifies a service within a host l Many standard port numbers are pre-assigned time of day 13, ftp 21, telnet 23, smtp 25, http 80 see /etc/services on workstation for list of all assigned ports l IP address + port number = "phone number“ for service Networking Basics

6. COMP201 Topic 14 / Slide 6 l protocols : rules that facilitate communications between machines l Examples: l HTTP: HyperText Transfer Protocol l FTP: File Transfer Protocol l SMTP: Simple Message Transfer Protocol l TCP: Transmission Control Protocol l UDP: User Datagram Protocol, good for, e.g., video delivery) l Protocols are standardized and documented So machines can reliably work with one another Networking Basics

7. COMP201 Topic 14 / Slide 7 l Client-Server interaction l Communication between hosts is two-way, but usually the two hosts take different roles l Server waits for client to make request Server registered on a known port with the host ("public phone number") Usually running in endless loop Listens for incoming client connections Networking Basics

8. COMP201 Topic 14 / Slide 8 l Client "calls" server to start a conversation Client making calls uses hostname/IP address and port number Sends request and waits for response l Standard services always running ftp, http, smtp, etc. server running on host using expected port l Server offers shared resource (information,database, files, printer, compute power) to clients Networking Basics

9. COMP201 Topic 14 / Slide 9 Using telnet to try out some services of servers: l Telnet assumes you want to connect to port 23 on the receiving host (port 23 is where the telnet server is listening) l However there is an optional argument after the hostname that allows you to connect to a different port l Try the following Get time: telnet time-A.timefreq.bldrdoc.gov 13 Get HTML page: telnet www.cs.ust.hk 80 and enter a GET command Many servers now refuse telnet connections due to security reasons. Networking Basics

10. COMP201 Topic 14 / Slide 10 Outline l Outline n Networking basics –IP addresses, ports, protocols, client-server interaction n Socket-level programming –Writing a client –Writing a server –Example: writing your own icq n Communicating with web servers –Retrieving information –Sending information

11. COMP201 Topic 14 / Slide 11 Socket-Level Programming l java.net.Socket is an abstraction of a bi-directional communication channel between hosts l Send and receive data using streams ClientServer OutputStream InputStream OutputStream l Next: n How to write a client n How to write a server

12. COMP201 Topic 14 / Slide 12 l To write a client Create a new Socket with hostname and port number of the connection Socket s = New Socket(String hostName,int portNumber); Call s.getOutputStream() and s.getInputStream() to get streams for sending and receiving information to/from the server n Need to learn protocol used to communicate –Know how to properly form requests to send to server –Know how to interpret the server’s responses Writing Clients

13. COMP201 Topic 14 / Slide 13 Writing Clients l SocketTest: Makes a socket connection to the atomic clock in Boulder, Colorado, and prints the time that the server sends. try { Socket s=new Socket("time-A.timefreq.bldrdoc.gov",13); InputStream inStream = s.getInputStream(); Scanner in = new Scanner(inStream); // read from in } catch (IOException e) { e.printStackTrace(); }

14. COMP201 Topic 14 / Slide 14 Writing Servers l To write a server Create a new java.net.ServerSocket with a port number to listen on the port ServerSocket s = New ServerSocket( portNumber); Use accept() to listen on the port. accept () returns a socket incoming when a client calls Socket incoming = s.accept(); Call incoming.getOutputStream() and incoming.getInputStream() to get streams for sending and receiving information to/from the client

15. COMP201 Topic 14 / Slide 15 Writing Servers l Example: Echo server ServerSocket s = new ServerSocket(8189); Socket incoming = s.accept( ); InputStream inStream = incoming.getInputStream(); OutputStream outStream = incoming.getOutputStream(); Scanner in = new Scanner(inStream); PrintWriter out = new PrintWriter (outStream, true); out.println( "Hello! Enter BYE to exit." ); … EchoServer.java

16. COMP201 Topic 14 / Slide 16 A side note l Most machines in CSD now refuse socket connections due to security considerations. l However, you can n Run a sever on any one of lab 4 machines and connect to the server from another lab 4 machine. n Run severs on one of scpu1-14 and connect to the server from others or from the PC network.

17. COMP201 Topic 14 / Slide 17 l Multithread server: starts a separate thread for each connection. public class ThreadedEchoServer { public static void main(String[] args ){ try{ int i = 1; ServerSocket s = new ServerSocket(8190); while (true){ Socket incoming = s.accept( ); System.out.println("Spawning " + i); Runnable r= new ThreadedEchoHandler(incoming, i); Thread t = new Thread(r); t.start(); i++;} } catch (IOException e) …. //ThreadedEchoServer.java Writing Servers

18. class ThreadedEchoHandler implements Runnable { public ThreadedEchoHandler(Socket i, int c) { incoming = i; counter = c; } public void run() { try {inputStream inStream = incoming.getInputStream(); OutputStream outStream= incoming.getOutputStream(); Scanner in = new Scanner(inStream); PrintWriter out = new PrintWriter(outStream, true); out.println( "Hello! Enter BYE to exit." ); boolean done = false; while (!done && in.hasNextLine()) { String line = in.nextLine(); out.println("Echo: " + line); if (line.trim().equals("BYE")) done = true; } …} private Socket incoming; private int counter; }

19. COMP201 Topic 14 / Slide 19 l A more interesting example n ICQServer.java – A simple server that listens on port 7777. –Connect two clients so that they can talk to each other. –Can handle more than one pairs. n ICQClient.java –Allows user to connect to ICQServer and have one-to-one conversation with partner Servers & Client

20. public class ICQServer { public static void main(String[] args ) { try { // create server socket to listen at port 7777 ServerSocket s = new ServerSocket(7777); // infinite loop: server for ever while (true ) { // first client Socket client1 = s.accept( ); PrintWriter out = new PrintWriter (client1.getOutputStream(),true); out.println("No partner available yet. Please wait."); Socket client2 = s.accept( ); // second client // start a ICQ Session for the two client Runnable r = new ICQSession(client1, client2); Thread t = new Thread(r); t.start() } } catch (Exception e) { System.out.println(e); } }}

21. class ICQSession implements Runnable { public ICQSession(Socket c1, Socket c2) { client1 = c1; client2 = c2; } public void run() { // new thread for chanel from client 1 to client 2 Runnable c1 = new Chanel( client1, client2); Thread t1 = Thread(c1); t1.start(); // new thread for chanel from client 2 to client 1 Runnable c2 = new Chanel( client2, client1); Thread t2 = Thread(c2); t2.start(); } private Socket client1; private Socket client2; }

22. class Chanel implements Runnable { public Chanel(Socket f, Socket t) { from = f; to = t; } public void run() { try { // open stream so that server can get messages from // "from“ and forward them to "to" Scanner in = new Scanner(from.getInputStream())); PrintWriter out = new PrintWriter (to.getOutputStream(),true); out.println("Connection established.");

23. while (true) { // message from "from" String str = in.nextLine(); // message to "to" out.println(str +"\r"); if (str.equals("quit")) {to.close(); from.close(); break; } } } catch (Exception e) { System.out.println(e); } } private Socket from; private Socket to; }

24. COMP201 Topic 14 / Slide 24 Outline l Outline n Networking basics –IP addresses, ports, protocols, client-server interaction n Socket-level programming –Writing a client –Writing a server –Example: writing your own icq n Communicating with web servers –Map –Retrieving information –Sending information

25. COMP201 Topic 14 / Slide 25 Map data structure stores key/value pairs. You can find a value if you provide the key. Java library supplies two general-purpose implementations for maps: HashMan and ThreeMap. Both classes implement the Map interface. For example, You can store a table of employee records, where the keys art the employee IDs and the values are Employee objects. Ex. Map staff = new HashMap (); Employee harry = new Employee(“Harry Hacker”); staff.put(“987-98-9996”, harry); To retrieve e = staff.get(“987-98-9996”); //get harry l Keys must be unique. If you call the put method wice with the same key, then the second value replaces the first one. The remove method removes an element with a given key. The size method returns the number of entries in the map. Maps

26. COMP201 Topic 14 / Slide 26 l The collections framework does not consider a map itself as a collection. However, you can obtain views of the map. There are three views: the set of keys, the collection of values ( which is not a set), and the set of key/value pairs. The key and key/value pairs form a set because there can be only one copy of a key in a map. l The following methods return these three views. : Set keySet(), Collection values(), Set > entrySet( ) l If you want to look at both keys and values, then you can avoid value lookups by enumerating the entries: for (Map.Entry entry : staff.entrySet()) { String key = entry.getkey(); employee value = entry.getValue(); //do something about key, value } // MapTest.java Maps

27. COMP201 Topic 14 / Slide 27 l Reason for communicating with web servers n To retrieve/send information l Need to indicate location of resource n URL stands for Uniform Resource Locator –Neat scheme for uniquely identifying all kinds of network resources n Basic form : –http://www.cs.ust.hk/~liao/comp201/index.html –ftp://ftp.cs.ust.hk/pub/liao/teach/201/codes/HttpTest/HttpTest.java –file:/MyDisk/Letters/ToMom2-11-98 –Protocols include files, http, ftp, gopher, news, mailto, etc. Communicating with web servers

28. COMP201 Topic 14 / Slide 28 Class java.net.URL provides basic connection to URLs Open connection with java.net.URL URL url = new URL(“http://www.cs.ust.hk/~liao/comp201/index.html”);http://www.cs.ust.hk/~liao/comp201/index.html To fetch contents of resource, use openStream method of URL, which returns a InputStream InputStream in = url.openStream(); Can nest the InputStreams with other Java streams to retrieve contents For additional information, use java.net.URLConnection URLConnection connection = url.openConnection(); Communicating with web servers

29. COMP201 Topic 14 / Slide 29 Steps for working with java.net.URLConnection 1. Set properties of connections: setDoInPut(true); //default setDoOutPut(true); for sending information to the server … Make connection: connection.connect(); n Query header information: getContentType, getContentLength,getContentEncoding, getDate, getExpiration, getLastModified 1. getInputStream for reading and getOutputStream for writing Communicating with web servers

30. COMP201 Topic 14 / Slide 30 Retrieving Information l URLConnectionTest.java URL url = new URL(urlName); URLConnection connection = url.openConnection(); connection.connect();//connect to the remote resource // print header fields Map > headers = connection.getHeaderFields(); for (Map.Entry > entry : headers.entrySet()) { String key = entry.getKey(); for (String value : entry.getValue()) System.out.println(key + ": " + value); }

31. COMP201 Topic 14 / Slide 31 // print convenience functions System.out.println("----------"); System.out.println("getContentType: " + connection.getContentType() ); System.out.println("getContentLength: " + connection.getContentLength() ); System.out.println("getContentEncoding: " + connection.getContentEncoding() ); …. Retrieving Information

32. COMP201 Topic 14 / Slide 32 // print first ten lines of contents Scanner in=new Scanner (connection.getInputStream())); for (int n = 1; in.hasNextLine() && n <= 10; n++) System.out.println(in.nextLine()); if (in.hasNextLine()) System.out.println("..."); Retrieving Information

33. COMP201 Topic 14 / Slide 33 Sending Information l Web servers receive information from clients using either GET or POST l GET requests are requests made by browsers when the user types in a URL on the address line, follows a link from a Web page, or makes an HTML form that does not specify a METHOD. l POST requests are generated when someone creates an HTML form that specifies METHOD="POST" l Examples: l http://maps.yahoo.com/py/maps.py: python, perl, http://maps.yahoo.com/py/maps.py … l http://www.census.gov/ipc/www/idbprint.html: http://www.census.gov/ipc/www/idbprint.html

34. COMP201 Topic 14 / Slide 34 Sending Information l An requested CGI (common gateway interface) script is called to process the information received and produces an HTML page to be send back to the client l CGI scripts usually written in C, Perl, shell script. (Out of the scope of this course.) l Will discuss servlets, Java alternative to CGI scripts

35. COMP201 Topic 14 / Slide 35 l Our task: Write java program to communicate with CGI scripts n The way we send parameters to a CGI script depends on –The way a CGI script receives parameters –The parameters that CGI scripts expects Sending Information

36. COMP201 Topic 14 / Slide 36 l Send information to CGI script using GET l Attach parameters to the end of URL http://host/script?parameters l Separate parameters using “&” and encode parameters as follows to avoid misinterpretation (URL encoding) l Replace space with “+” l Replace each non-alphanumeric character with “%” followed by the hexadecimal code of the character “Mastering C++”  “Mastering+C%2b%2b” Disadvantage: long parameter string, might exceed limits of browsers. GetTest.java Sending Information

37. COMP201 Topic 14 / Slide 37 l Sending information to CGI script using POST: Open URLConnection and send parameter using a stream l Open a URLConnection: URL url = new URL(“http:/host/script”); URLConnection connection = url.openConnection(); l Set up connection for output: connection.setDoOutput(true); l Get a stream for sending data: PrinterWriter out = new PrintWriter (connection.getOutputStream()); Sending Information

38. COMP201 Topic 14 / Slide 38 l Send parameters boolean first = true; for (Map.Entry pair : nameValuePairs.entrySet()) { if (first) first = false; else out.print('&'); String name = pair.getKey(); String value = pair.getValue(); out.print(name); out.print('='); out.print(URLEncoder.encode(value, "UTF-8")); } PostTest.java Sending Information

39. COMP201 Topic 14 / Slide 39 l Note: URLEncoder: Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x- www-form-urlencoded MIME ( Multipurpose Internet Mail Extensions ) format. The World Wide Web Consortium Recommendation states that the UTF-8 encoding scheme should be used.World Wide Web Consortium Recommendation PostTest.java Sending Information