1. Uncovering Algorithms: Looking Inside the Facebook News Feed Motahhare Eslami, Kevin Hamilton, Karrie Karahalios, Cedric Langbort, Aimee Rickman, Christian Sandvig

5. “The preferential display of our flights, and the corresponding increase in our market share, is the competitive raison d’être for having created the [SABRE] system in the first place.” Robert Crandall, CEO American Airlines

6. SOURCE: ReadWrite

7. 1.How can research on algorithms proceed without access to the algorithm? Then, initial results from a study about the Facebook news feed, investigating: 2.What is the algorithm doing for a particular person? 3.How should we usefully visualize it? 4.How do people make sense of the algorithm? Questions

8. Social Science Audits 8

11. Our proposal: The Algorithm Audit 11 Five illustrative research designs

12. Get the Code

13. Ask the Users

14. Scrape Everything

15. Sock Puppets

16. Collaborative Audit

17. Our Audit

18. How aware are users of the role of algorithms and filters in their news sources?

19. Kevin Lynch, “Wayfinding”

20. Interviewing/surveying users Exposing hidden algorithmic processes with social media prompts using the API and studying the effects of knowledge on use Working with users to try and deduce algorithmic processes and the design rationale behind them Our Approach

22. FeedVis

24. Aware 37.5% Unaware/Uncertain 62.5%

29. Levels of Awareness

30. Paths to Awareness “but some people have...I mean I have 250 friends but some people have thousand... [and if it wants to show everything], there is no way, it couldn't... it crashes your computer [laughing].” (P24) “… Plus like the phone newsfeed is different from the laptop newsfeed sometimes like I look at something on my phone but then when I go online, there's a ton more. So, it's also “where” you look.” (P26) “I read this a while ago, a year or two ago. That, all these keys of things that you have to do to get into people news feed.” (P21)

31. Unaware “I bet it would be on my news feed. I probably would catch [it] at some point during the day.” (P30) “... and probably I don't scroll down enough but you know [you] cannot always scroll down so much, you just go like a few posts and that’s it.” (P9) “I don't know for sure [...] because I know there are some friends I guess, it seems like I don't see anything by them very often. I don't know if they really use Facebook that much or […]" (P36) “My wife also uses Facebook and she has occasionally asked me if I saw a post by her friend and I will not have seen it and I don't know why I didn't see it.” (P36)

32. Exploring Awareness Factors Membership Duration Seen Content/Total Content Friendship Network Size Usage Frequency Activity Level (Poster, Light Poster, Listener) Facebook Page/Group Management Newsfeed Parameter Manipulation (top/recent stories, block people, hide posts)

33. Reactions to FeedVis

34. “So do they actually hide these things for [from] me? [mentioning to the unseen stories] Heeeeeeeeeeeeeeeey!” (P1) “It's kind of intense because... have you seen the movie "The Matrix"? It's kind of waking up in the Matrix in a way...I mean you have what you think as your reality of like what they choose to show you.”(P19) What the hell Facebook? [after scrolling Content View] Well, I’m super frustrated… (P3) Initial Surprise (In seen/not seen content-centric view)

35. “So, there is some algorithm or something or some rules to choose these white things that would not appear to me? [....]It’s very interesting. I never know that FB really hides something! So they are doing some data mining or machine learning thing?” (P1)

36. “For now, I cannot really understand how they categorize these people. Actually this is my brother [pointing to sometimes seen] and actually he needs to be here [pointing to almost seen]. “(P1) (In people-centric view) “I think she needs support for that [mentioning to a not shown story]; if I see it, then I will say something [to support her].” (P8) (In content-centric view)

38. Folk Theories

39. After Follow-up

40. After Learning about the Algorithm, People Changed their Behavior

41. Reading Behavior Changing interaction behavior Using the most recent feature Changing news feed settings (ex. hiding posts) Posting Behavior Relationship Changes Unfriending people

42. “After our discussion, I actually went back and started experimenting a little with the newsfeed and discussing with some friends on ways to streamline what I was receiving. Since then, I’ve become more interested in checking my Facebook because it does not seem as cluttered with random information I have no interest in.” (P10)

44. Conclusion A Question, an Admonishment, and a Call to Arms What do you users need to know? Transparency alone is not enough! Create infrastructure for algorithm auditing!