1. Department of Computer Science Internet Performance Measurements using Firefox Extensions Scot L. DeDeo Professor Craig Wills

2. Department of Computer Science Motivation There exists an API within the Firefox web browser that allows for interacting with its downloading and processing of websites Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?

3. Department of Computer Science Outline Introduction to Firefox Extensions  Description  Capabilities Page Stats Extension  Overview  Approach and paths taken  Demo Summary

4. Department of Computer Science Introduction Firefox Extensions are add-ons that integrate directly into the Firefox Web browser. Function through open source API that allows for manipulating both the content being received by the web browser as well as the graphical outlook.

5. Department of Computer Science Capabilities Graphical add-ons (Use the XML User Interface Language – XUL)  Allows for changing the existing Firefox overlay, adding new buttons, menus, and windows.  E.g. Google toolbar allows for a toolbar to be inserted directly into Firefox. Middleware modifications (Cross Platform Component Object Module – XPCOM Interfaces - Accessible through JavaScript or C++)  Allows for changing networking properties, modifying / blocking data requests / responses, etc.  E.g. Fasterfox allows you to tweak many network and rendering settings such as simultaneous connections, pipelining, cache, DNS cache, and initial paint delay.  E.g. AdBlock and AdBlock plus allow for use of a pre-formed list or to customize a list of sites to block downloads from.

6. Department of Computer Science Capabilities Combination of Graphical add-ons and Middleware modifications can yield new applications that exist within Firefox.  E.g. Gmail Space – Allows users to take advantage of Gmail’s 2 GB quota by providing a graphical interface to easily transfer files. Fixing bugs in other sites  Allows for changing sites html, style sheets, and inserting JavaScript. E.g. Style sheets can be modified to change the background color and text of a site (change to black on white instead of yellow on white).

7. Department of Computer Science Website Request / Response Breakdown 1. Location Changes – Request for container (main) page goes out. 2. Response for container (main) page is received and processed. 3. Additional requests are spawned in response to the processed page: CSS, JavaScript, images, etc. 4. Responses come in and are processed and may / may not spawn additional requests. Container Page CSS JavaScript Jpeg Spawns JavaScript Spawns Browser location set to: http://www.scotd.com/index.php http://www.scotd.com/index.php?id=all http://www.scotd.com/scot.css http://www.scotd.com/background/valley2k.jpg Spawns

8. Department of Computer Science Outline Introduction to Firefox Extensions  Description  Capabilities Page Stats Extension  Overview  Approach and paths taken  Demo Summary

9. Department of Computer Science Page Stats Extension Measures the HTTP requests associated with the load of a webpage. Page Measurements Include: # of Request Request submit time (msec) Finish Time (msec)HTTP Response Code Size (bytes) content type URL Page Summary Includes: URL total page size (bytes) total time (msec) total # of requests # dead requests Capabilities:  Control input and output with a graphical interface  Output to a log file  Run in a batch mode and visit a list of sites

10. Department of Computer Science Page Stats Extension Usage: 1. Graphical Interface found under the tools menu in Firefox - “Site Measurements” Ability to enable / disable measurement listeners, log file, and select a list of sites to visit. Also allows for saving and clearing the text on the measurement window. Provides a summary of all sites visited.

11. Department of Computer Science Page Stats Extension Usage: 2. Batch mode – Invoked by a configuration file specified on the command line (firefox pagestatsconfig=“/home/sdedeo/config.txt”). Parameters: measurement: on log: /home/sdedeo/log.txt sites: /home/sdedeo/sites.txt exitOnFinish: true requestTimeout: 8000 pageTimeout: 45000  Allows for easily measuring pages in bulk  Pages are still fully rendered

12. Department of Computer Science Approach and paths taken Began in January 2006 based on work completed by Paul Timmins  Included initial work of setting up an extension with ideas on how to make measurements Initial version based on an XPCOM component called nsiWebProgress which reports to the browser:  State, Progress, Location, Status, and Security changes

13. Department of Computer Science Approach and paths taken First Approach Issues  nsiWebProgress progress notification only occurs on text based files (E.g. html, CSS, JavaScript). Images and other static files are omitted.  No way to measure the time the request is submitted  Occasionally the progress notification gives an unknown length of a file  If a site was visited more then once, caching effects skew measurements Second Approach – nsiHTTPChannel  Intercepts requests browser sends – “http-on-modify-request”  Intercepts responses before processing – “http-on-examine-response”  Clears system cache between sites in order to maintain consistent measurements

14. Department of Computer Science Approach and paths taken Second Approach (Continued)  nsiHTTPChannel provides no state notifications Solved using nsiWebProgress state change to detect state stopped / finished Probe to insure all requests that have gone out have come back (nsiWebProgress state stop returned before all images received)  Timeout Mechanism Two Timeouts:  Max time between receiving responses – defaults to 8 seconds  Max time for a page – defaults to 45 seconds

15. Department of Computer Science Approach and paths taken Sites with Issues:  http://www.cnn.com routinely sends requests for images in which no response comes. These are JavaScript invoked images on mouseovers http://www.cnn.com This problem is solved by the request timeout and the requests are labeled as ‘dead’  http://my.netscape.com uses a JavaScript based redirect: ‘document.location = http://my.netscape.com/index2.psp’ which when used in batch mode was causing the extension to move onto the next site and process two sites simultaneously http://my.netscape.comhttp://my.netscape.com/index2.psp This was solved by giving a one milliseconds delay before switching sites after a site’s results were processed  http://www.dallasnews.com sends an image banner which receives an update once every two seconds http://www.dallasnews.com This was solved with the max site timeout

16. Department of Computer Science Demo Launch Firefox

17. Department of Computer Science Outline Introduction to Firefox Extensions  Description  Capabilities Page Stats Extension  Overview  Approach and paths taken  Demo Summary

18. Department of Computer Science Summary Firefox extensions are a powerful tool that allow for:  Measuring and manipulating how a page is downloaded and processed  Adding additional features, both graphical and background Building Extensions:  Firefox 1.5 enables much easier testing of extensions  XPCOM API is still in the beta stages and certain libraries must be used with caution since they change  Documentation available is still minimal, but the development community is growing at a fast pace

19. Department of Computer Science Summary Can this functionality be exploited to allow us not just to improve the web surfing experience, but to understand it under realistic conditions?  The pagestats extension shows one way we can create quantitative measurements for a website  Pagestats can be used in conjunction with other extensions, such as adblock and fasterfox, to compute a measurement of effectiveness

20. Department of Computer Science Questions / Comments Scot L. DeDeo Professor Craig Wills Page Stats Extension: http://www.wpi.edu/~sdedeo/pagestats Related Sites: https://addons.mozilla.org/firefox/extensions/ http://www.xulplanet.comhttp://www.xulplanet.com (XUL and XPCOM API Reference)