1. Dinei Florencio and Cormac Herley WEIS’ 2011

2. We are curious about sex We are curious about politics We are curious about cybercrime And we satisfy some of these curiosities with surveys …

3. However… HOW BIG IS CYBERCRIME? “The spoils of cyber crime almost double in 2009. As a whole, losses totaled $560m," Patrick Peterson, Cisco Fellow [1]. Cyber crime costs corporate America $10 billion every year!" [2]. Damage caused by cyber-crime is estimated at $100 billion annually," Kilian Strauss, Organization for Security and Cooperation in Europe (OSCE) [3]. Cyber-crime revenues are worth approximately $1 trillion," Edward Amoroso, CSO, AT&T (written testimony to the US Senate, March 17, 2009).

4. Surveys and errors… In a 1983 Federal Reserve survey, a single incorrectly recorded answer inflated the estimate of US household wealth by 1 trillion. In a 2006 FTC survey of ID theft, two respondents were excluded. Inclusion would have increased the estimate by 37 billion (3X)

5. Political Surveys Vote intention surveys seem reliable… Republican/Democrat What happens if a respondent lies?

6. Set membership vs Numeric surveys How many other lies to compensate a DEM/REP lie? How to compensate for a large, positive number, lie? How to keep the large number, when truthful?

7. Sex and Lies Particularly interesting sex survey: How many sexual partners of opposite sex have you had? SURVEY RESULTS: man report 2.5 to 9 times more lifetime partners than woman do. On a closed population, with M~W, the average should be similar for M/W. So, why the difference?

8. Sources of errors in Surveys Response error Non-response error Sampling error X Y R

9. Example: Gambling Survey

10. Sex and lies SURVEY RESULTS: man report 2.5 to 9 times more lifetime partners than woman do. Possible explanations: Highly skewed distribution: (e.g., prostitution). Could explain some, but not all. Age difference of couple: Would increase the difference… Different definition of “partner”, or “sex”... Biased lies Men exaggerate Women reduce

11. Sex and Lies “the gender discrepancy appears to be driven primarily by [a few men reporting] large, “round” number of partners”. [Wiederman, 1997] On a prostitution survey, a respondent claimed 5401 partners in the last 6 months… [Brewer at al., 2000] So, could survey estimates of number of sexual partners be off by 5X due to a few liars??

12. The difficult of sampling heavytail distribution Cybercrime surveys Mean/median: 4 to 10 => highly skewed distribution. US Weatlh surveys US wealth mean/median: 4.3

13. Fraction of the wealth controlled by segments of the population [25]. The top 1% and bottom 90% each control about one third of the wealth.

14. Instances of sample mean for a Pareto ( = 2:32; i.e., US wealth) distribution. There are 100 trials each with N = 1000 samples.

15. Cybercrime surveys Victims only a fraction of population (phishing: ~ 0.4%) On a 5000 people survey, 20 are victims. Mean/median higher than for US wealth. Need to sample more & better. Uniform sample may require well over 1 million respondents! Possibility of exaggeration on answers Risk of contamination from the 99.6% of non-victims

16. Conclusions “You should never trust user input”