1. Building HTTP clients in PHP

2. A PHP package for sending HTTP requests and getting responses A PHP package for handling HTTP requests/responses is available It is called HTTP_Request and is part of the PEAR repository of PHP extensions and applications -- see http://pear.php.net/ The HTTP_Request documentation is here: http://pear.php.net/manual/en/package.http.http-request.php PEAR uses the object-oriented programming paradigm which is supported by PHP Before looking at HTTP_Request, we will review a few details of OOP in PHP

3. OOP in PHP (contd.) The object model in PHP was rewritten for PHP 5 The PHP manual contains two main section on OOP: –Chapter 18. Classes and Objects (PHP 4) –Chapter 19. Classes and Objects (PHP 5) A full treatment of OOP in PHP is beyond our scope here

4. Object-oriented programming in PHP Example class and its usage <?php class widget { var $x; function say_hello() {echo "Hello, World!";} function set_x($value) {$this->x = $value;} function get_x() {return $this->x;} } $thing1 =& new widget; $thing1->say_hello(); $thing1->set_x(98); $y = $thing1->get_x(); echo " y is $y"; ?>

5. Example HTTP client program in PHP <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request('http://www.rte.ie/'); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); echo $contents; } ?> Note we have not handled fact that some URLs on RTE site are relative Thus, output (top image) not quite right; should be as in bottom inage

6. HTTP_Request package We will now examine details of the HTTP_Request package Information on the package is here: http://pear.php.net/package/HTTP_Request/docs/ Some end-user documentation is available here: http://pear.php.net/manual/en/package.http.http-request.php

7. HTTP_Request methods Current description of the HTTP_Request class is here: http://pear.php.net/package/HTTP_Request/docs/latest/apidoc/HTTP_Request-1.2.4/HTTP_Request.html (This may become outdated as new versions of the class are produced) The list of methods provided by the class appears to be HTTP_Request setMethod setURL setHttpVer addHeader addCookie removeHeader addQueryString addRawQueryString addPostData addRawPostData addFile setBasicAuth setProxy sendRequest getResponseCode getResponseHeader getResponseCookies getResponseBody

8. HTTP_Request (contd.) The set of methods provided by HTTP_Request corresponds to the structure of requests and responses defined in the HTTP protocol

9. Structure of HTTP requests/responses Requests: a three-part structure request line (Method URL[+query] HTTP-version) headers body Responses: also a three-part structure status line (HTTP-version NatLangPhrase Response-Code) headers body

10. Structure of HTTP requests and corresponding HTTP_Request methods Request structure request line (Method URL[+query] HTTP-version) headers body HTTP_Request methods Methods for manipulating the Request Line setMethod setURL setHttpVer addQueryString addRawQueryString Methods for manipulating the Headers addHeader addCookie removeHeader Methods for manipulating the body addPostData addRawPostData addFile

11. cs 4408 got here on 1 nov 2005/

12. Structure of HTTP responses and corresponding HTTP_Request methods Response structure status line (HTTP-version NatLangPhrase Response-Code) headers body HTTP_Request methods Methods for reading the status line getResponseCode Methods for reading the Headers getResponseHeader getResponseCookies Methods for reading the body getResponseBody

13. Creating a request object Method for creating a request object HTTP_Request Format of call: HTTP_Request HTTP_Request( $url $url [,array() $params]) Params is asssociative array which can include: –method - Method to use, GET, POST etc –http - HTTP Version to use, 1.0 or 1.1 –user - Basic Auth username –pass - Basic Auth password –proxy_host - Proxy server host –proxy_port - Proxy server port –proxy_user - Proxy auth username –proxy_pass - Proxy auth password –timeout - Connection timeout in seconds.

14. Creating a request object (contd.) Example usages of method for creating a request object $req =& new HTTP_Request(); $req =& new HTTP_Request("http://www.rte.ie/"); $req =& new HTTP_Request("http://www.rte.ie/", array(method=>'GET',http=>'1.1')); In the first example, no aspect of the new request is specified; they can all be specified later, using other methods, before the request is sent to the server In the second example, the URL is specified; other aspects can be specified later, using other methods In the third example, the method, the URL and the HTTP version are specified; other aspects can be specified later, using other methods

15. Sending a request object Method for sending a request object sendRequest Format of call: mixed sendRequest() Method returns: –a PEAR error message if there is an error, –true otherwise Example usage of method for creating a request object Example 1: $request =& new HTTP_Request("http://www.rte.ie/"); $request->sendRequest();

16. Sending a request object (contd.) HTTP_Request is an object class which extends a base class called the PEAR base class The PEAR base class contains a range of methods -- see Chapter 26 of the PEAR manual, which is available at http://pear.php.net/manual/index.php One of these is the isError method which checks for a PEAR_Error object Format of call boolean PEAR::isError (mixed $data [, mixed $msgcode]) It can be used to see whether an error is produced when a HTTP request is sent Example 2 $request =& new HTTP_Request("http://www.rte.ie/"); if ( ! PEAR::isError( $request->sendRequest()) ) {......... }

17. Revisiting the example client program in PHP <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request('http://www.rte.ie/'); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); echo $contents; } ?> Note we have not handled fact that some URLs on RTE site are relative Thus, output not quite right -- some image URLs are not quite right

18. Revisiting, again, the example client program <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request('http://www.rte.ie/'); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); $contents= str_replace('src="/', 'src="http://www.rte.ie/', $contents); echo $contents; } ?> We have handled some of the relative URLs on RTE site But, output still not quite right

19. Revisiting, yet again, the example client program <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request('http://www.rte.ie/'); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); $contents= str_replace('src="/', 'src="http://www.rte.ie/', $contents); $regexp = '%src="(?!http://)%'; $contents= preg_replace($regexp, 'src="http://www.rte.ie/', $contents); echo $contents; } ?>

20. A utility which shows all data from incoming requests is here: http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest.php It is defined as follows: <?php echo " SERVER variables: "; foreach ($_SERVER as $name => $value) { echo "$name = $value "; } echo " GET variables: "; foreach ($_GET as $name => $value) { echo "$name = $value "; } echo " POST variables: "; foreach ($_POST as $name => $value) { echo "$name = $value "; } echo " COOKIE variables: "; foreach ($_COOKIE as $name => $value) { echo "$name = $value "; } ?>

21. When called by a MSIE browser

22. Consider this "browser" calling the same showRequest.php It is a PHP program at http://cosmos.ucc.ie/~jabowen/cs4408/myBrowser.php It is implemented as follows: <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request("http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest.php"); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); echo $contents; } ?>

23. Output from showRequest.php when called by "browser"

24. Sending headers

25. A modified "browser" It is a PHP program at http://cosmos.ucc.ie/~jabowen/cs4408/myBrowser2.php It is implemented as follows: <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request("http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest.php"); $req->addHeader('ACCEPT-LANGUAGE','en-ie'); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); echo $contents; } ?>

26. Output from showRequest.php when called by "browser" ACCEPT_LANGUAGE header now appears

27. Handling cookies

28. Modified utility to show all data from incoming requests: http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest2.php <?php ob_start(); setcookie('dummyCookie','baconAndEggs'); ob_end_flush(); echo " SERVER variables: "; foreach ($_SERVER as $name => $value) { echo "$name = $value "; } echo " GET variables: "; foreach ($_GET as $name => $value) { echo "$name = $value "; } echo " POST variables: "; foreach ($_POST as $name => $value) { echo "$name = $value "; } echo " COOKIE variables: "; foreach ($_COOKIE as $name => $value) { echo "$name = $value "; } ?>

29. In first call by a MSIE browser, no cookie comes in request

30. In second call by a MSIE browser, a cookie is in request

31. A "browser" to read showRequest2.php It is a PHP program at http://cosmos.ucc.ie/~jabowen/cs4408/myBrowser3.php It is implemented as follows: <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request("http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest2.php"); $req->addHeader('ACCEPT-LANGUAGE','en-ie'); if (!PEAR::isError($req->sendRequest())) { $contents= $req->getResponseBody(); echo $contents; } ?>

32. cs 4408 got here on 4 nov 2005

33. In first call, no cookie comes in request

34. No cookie comes in 2nd call either

35. Why no cookie in 2nd request? We know that showRequest2.php sent a cookie to our "browser" when it sent a response to the first request Why did that cookie not come back in the 2nd request? Because our "browser" does not maintain a "cookie jar" and never sends cookies

36. Cookie "jars" A cookie "jar" is the mechanism used by a HTTP client to store cookies it receives from servers

37. In MSIE, the cookie jar is a directory on the hard- disk of the client machine; each cookie is file

38. In Netscape 8.0, the cookie jar appears to be a single file managed by an interface which allows you to see cookie properties

39. In Firefox, the cookie jar appears to be a single file managed by an interface which allows you to see cookie properties

40. Detecting cookies Before we implement a cookie jar in our web client, let's first make our "browser" report cookies that it receives from the server-side program

41. A "browser" to read showRequest2.php and which can show cookies and other headers that are sent by the server At http://cosmos.ucc.ie/~jabowen/cs4408/myBrowser4.php <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request("http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest2.php"); if (!PEAR::isError($req->sendRequest())) { $headers = $req->getResponseHeader(); echo " Headers "; foreach ($headers as $name => $value) { echo " $name = $value"; } echo " Cookies "; $cookies = $req->getResponseCookies(); foreach ($cookies as $fields) { foreach ($fields as $name => $value) { echo "$name = $value; "; } echo " "; } $contents= $req->getResponseBody(); echo " Body "; echo $contents; } ?>

43. Adding a cookie jar to our web client We could use a simple text file as our cookie jar Initially, let's have a simple jar, which stores only the names and values of cookies, without storing such attributes a expiry date, web-address, etc Let's call the file cookieJar

44. A "browser" to read showRequest2.php and which remembers and returns a cookie sent by the server At http://cosmos.ucc.ie/~jabowen/cs4408/myBrowser5.php <?php require_once "HTTP/Request.php"; $req = & new HTTP_Request("http://www.cs.ucc.ie/j.bowen/cs4408/resources/showRequest2.php"); $fp=fopen("tmp/cookieJar","r"); $cookie = fread($fp,100); fclose($fp); if ($cookie) { $pieces=explode("=",$cookie); $req->addCookie($pieces[0],$pieces[1]); } if (!PEAR::isError($req->sendRequest())) { echo " Non-cookie Headers received by Browser "; $headers = $req->getResponseHeader(); foreach ($headers as $name => $value) { echo " $name = $value"; } echo " Cookies received by browser "; $cookies = $req->getResponseCookies(); $fp=fopen("tmp/cookieJar","w"); foreach ($cookies as $fields) { foreach ($fields as $name => $value ) { echo "$name = $value; "; } echo " "; fwrite($fp,"$name=$value"); } fclose($fp); $contents= $req->getResponseBody(); echo " Message Body rec'd by Browser "; echo $contents; } ?>

45. Before 1st call, cookieJar is empty; so, no cookie is sent

46. However, after the request the cookieJar is not empty, as we can see in the vi session below

47. Since cookieJar is no longer empty, a cookie is sent the next time we run our "browser"; its arrival is reported by server-side program