1. Measuring wireless network usage with the Experience Sampling Method Tristan Henderson, Denise Anthony, David Kotz Dartmouth College Tristan Henderson, Denise Anthony, David Kotz Dartmouth College

2. WinMee, April 3 2005Measuring wireless network usage with ESM Motivation Measuring wireless networks is useful deployment, protocol development, application development, mobility modeling, etc... So lots of people have done it Network-side measurements of lots of WLANs: IBM, UNC, SIGCOMM, Georgia Tech, Stanford, UCSD, Dartmouth,... Network-side measurement only gets you so far Tells you what happens, but not why it happens To know why, need to ask wireless users themselves But reliable polling can be difficult

3. WinMee, April 3 2005Measuring wireless network usage with ESM The Experience Sampling Method A psychological diary method subjects self-report their own experiences potentially-fewer biases since no interviewer involved signal subjects at random times and ask them to report timely reports means users dont forget their experiences interval-contingent: subjects report at regular intervals signal-contingent: subjects report when an event occurs ESM reports: subjects fill out questionnaires questionnaires ask about current activities, conditions, feelings finite time period, e.g., seven signals per day over seven days

4. WinMee, April 3 2005Measuring wireless network usage with ESM Experimental setup - users Dartmouth WLAN 560 Cisco/Aruba 802.11a/b/g APs complete campus coverage ongoing monitoring since 2001 (syslog, SNMP, tcpdump, CDR) 30 subjects all undergraduates, 15 male, 15 female all owned wireless laptops 23 Windows, 7 Macs recruited via website offered $100 each subjects provided conflict times: times not to be signaled

5. WinMee, April 3 2005Measuring wireless network usage with ESM Experiment setup - technical 30 participants, 30 alphanumeric pagers, 30 notebooks Motorola Bravo 406-512MHz pagers considered PDAs or 802.11 devices but wouldnt work off-campus or when network down Notebook: signal- and event-contingent questionnaires different questionnaires depending on type of alert questions incl: devices/applications in use, location, current task 7 day study, 7 signal-, 3 event-contingent alerts/day signal-contingent alerts at random times (excluding conflicts) minimum 45-minute interval between alerts monitored subjects wireless MAC addresses and sent event-contingent alerts if certain events occurred high throughput, high errors, ping-ponging, association with busy AP

6. WinMee, April 3 2005Measuring wireless network usage with ESM Wireless vs overall mobility Users were paged at up to 6 locations in a day, but were only on WLAN at 2 Do we need better coverage?

7. WinMee, April 3 2005Measuring wireless network usage with ESM Usage locations Wireless used more in homes, libraries % of alerts at location

8. WinMee, April 3 2005Measuring wireless network usage with ESM Applications in use Unsurprisingly, P2P was big bandwidth hog - our sample is representative of whole campus

9. WinMee, April 3 2005Measuring wireless network usage with ESM Wireless vs wired behaviour Wireless behaviour is multi-modal when using wireless laptop, more than one device in use More likely to be using cellphone and/or TV when on wireless Fewer high-bandwidth activities (filesharing, streaming media) on wireless than on wired [Wireless] tends to be a little bit slower than wired... I download a lot of music so that can be frustrating.

10. WinMee, April 3 2005Measuring wireless network usage with ESM Home vs non-home behaviour Both home and non-home: e-mail work-related web-browsing When at home: P2P, IM non-work-related web-browsing [Wireless] lets me move my study space to wherever I want it to be.

11. WinMee, April 3 2005Measuring wireless network usage with ESM Communication patterns Locality 83% of email/IM is local 86% of phone (analogue/cell/VoIP) is non-local VoIP 65% of subjects completely unaware of VoIP VoIP users did not use cell phones, and vice versa (but we have very little cellphone usage in general) If the quality and price of VoIP phones were the same as the cell phone, I think I would definitely use them.

12. WinMee, April 3 2005Measuring wireless network usage with ESM Lessons learned Dealing with subjects is hard work! choose time carefully avoid student examinations, vacations allow time to collect info (MAC addresses etc.) expect schedules to change students change conflict times with 5-minutes notice... one laptop broke and was replaced during the week explain parameters clearly one user went home (to a different state) but thought she could still be monitored on home WLAN automatic participant pre-screening we used face-to-face interviews could monitor wireless activity prior to participant-selection

13. WinMee, April 3 2005Measuring wireless network usage with ESM More lessons learned Consider all event-contingent alerts we thought it would be less intrusive to limit to 3/day ended up with very few event-contingent alerts ignore 45-minute alert interval interval meant we ignored some interesting wireless events Use a two-way signalling channel lots of pages failed, but we didnt know no real-time feedback users would complain, but only at the end of the day (when they hadnt received sufficient pages) two-way pagers? 802.11 devices inappropriate

14. WinMee, April 3 2005Measuring wireless network usage with ESM Related work Lots of ESM studies in psychological literature teacher job satisfaction youth happiness experiences of new parents Communications-based ESM studies Ubicomp - Intel Personal Server (Consolvo & Walker 2003) Virtual Environments (Gaggioli et al. 2003) Television-viewing habits (Kubey et al. 1986/1987/1990) Most studies are signal- or event-contingent Our study attempts to combine both

15. WinMee, April 3 2005Measuring wireless network usage with ESM Summary As well as measuring wireless networks, need to measure wireless network users ESM provides a technique for measuring user experiences allows us to see when users arent on the WLAN allows us to see more detail about what users are doing allows us to see usage of non-802.11 devices ESM studies are difficult to set up difficult to deal with users Future work: larger study that benefits from the lessons learned in this study