28. June Crowdsourcing – Challenges and Opportunities in Web Science Imaad Mohamed Khan

Toward a Learning Science for Complex Crowdsourcing Tasks Shayan Doroudi, Ece Kamar, Emma Brunskill and Eric Horvitz 28. June 2016Imaad Mohamed Khan

Introduction Training crowd sourced workers to perform complex tasks. Hypotheses about the different training methods were proposed by the authors. Experiments were performed to reach to conclusions about the different hypotheses. 28. June 2016Imaad Mohamed Khan

Introduction 28. June 2016Imaad Mohamed Khan

Task and Experiment Design The hypotheses were tested on a complex web search task. Two training tasks, X and Y, and five test tasks, namely, A, B, C, D and E were decided. All experiments were run on the Amazon Mechanical Turk. Workers received the conditions in a round robin fashion to balance the number of workers for each task. 28. June 2016Imaad Mohamed Khan

Experiment June 2016Imaad Mohamed Khan

Experiment 1 Results 28. June 2016Imaad Mohamed Khan

Experiment 1 Results 28. June 2016Imaad Mohamed Khan

Filtering Validation Tasks 28. June 2016Imaad Mohamed Khan

Experiment June 2016Imaad Mohamed Khan

Experiment 2 Results 28. June 2016Imaad Mohamed Khan

Conclusion 28. June 2016Imaad Mohamed Khan From experiment 1,expert examples were found to be the most effective form of training amongst various strategies. From experiment 2, high quality validated solutions can be potentially more efficient than expert examples. The authors conclude saying that similar experiments should be performed across a series of complex problem solving domains to reach to further conclusions.

Imaad Mohamed Khan13 Questions?

Psychological Maps 2.0: AWeb Engagement Enterprise Starting in London Daniele Quercia, Joao Paulo Pesce, Virgilio Almeida and Jon Crowcroft 28. June 2016Imaad Mohamed Khan

Introduction A psychological map is the subjective representation that each city dweller carries around in his/her head. A web game was built to put the recognizability of London’s street to the test. Areas with different economic and social patterns had different results in the recognizablility test. The experiment follows closely as possible to Stanley Milgram‘s experiment on New York in June 2016Imaad Mohamed Khan

Test for recognizability Random locations from Google Street View and tests users to determine in which subway location (or boroughs or region) the scene is. Data from 2255 users was collected and a collective recognizability map was built. The extent to which an area’s recognizability is explained by the area’s exposure to people. 1.2M Twitter messages, 224K Foursquare check-ins, 76.6M underground trips and 1.3M Flickr pictures in London. 28. June 2016Imaad Mohamed Khan

Game Design For each round, the game shows a player a randomly selected scene in London and ask him to guess the nearest subway station or borough or region. Points were given to each user for the right answer. 15 Points were also given for ‘I don’t know’ to reduce random guesses. Players were allowed to post their scores on social media, i.e. Facebook and Twitter. The player has to recognize 10 images in Greater London. 28. June 2016Imaad Mohamed Khan

Player Statistics 28. June 2016Imaad Mohamed Khan

Relative Recognizability 28 June 2016Imaad Mohamed Khan

Reasons for Recognizability 28 June 2016Imaad Mohamed Khan People recognize an area because they are exposed to it and because an area offers distinctive architecture. Milgram hypothesized the recognizability by R = f(C.D) where C is the centrality of population flow and D the social and architectural distinctiveness.

Recognizability and Exposure 28. June 2016Imaad Mohamed Khan

Recognizability and Exposure Pearson‘s product-moment correlation (r) between recognizability and exposure was calculated for all four classes. 28. June 2016Imaad Mohamed Khan

Recognizability and Well Being Firstly, each of the deprivation factors was correlated with recognizability. It was then correlated with social media data. From the correlation between the factors and social media data, boroughs with good living conditions tend to be more recognizable as well. (r=0.61) 28 June 2016Imaad Mohamed Khan

Discussion and Conclusion This research is related to early urban studies and also taps into research on „games with a purpose“ or „serious games“. The findings of this research can be used in planning urban interventions. This research can also lead to different experiments on the web with regards to games with a purpose. Finally, this research can be extended to cities beyond London and further conclusions can be drawn. 28 June 2016Imaad Mohamed Khan

25 Thank You