1. Picking the NYT Picks: Editorial Criteria and Automation in the Curation of Online News Comments Nicholas Diakopoulos University of Maryland, College Park – College of Journalism @ndiakopoulos | nickdiakopoulos.com | nad@umd.edu

4. “NYT Picks is the most popular comment queue. We spend a lot of time tweaking that and getting that right.” What are criteria for selection? How can we augment moderator capability to consider more comments?

5. Criteria from Literature Negative / Exclusion Personal attacks, profanity, abusive behavior Positive / Inclusion Internal Coherence Thoughtfulness Brevity / Length Relevance Fairness / Diversity Novelty Argument Quality Criticality Emotionality Entertainment Value Readability Personal Experience

6. Crowdsourcing Argument Quality Criticality Emotionality Entertainment Value Readability Personal Experience Internal Coherence Thoughtfulness Brevity / Length Relevance Fairness / Diversity Novelty RQ1: Do “NYT Picks” comments reflect positive editorial criteria identified in literature?

7. Automation Argument Quality Criticality Emotionality Entertainment Value Readability Personal Experience Internal Coherence Thoughtfulness Brevity / Length Relevance Fairness / Diversity Novelty RQ2: Can algorithmic approaches to assessing criteria be developed?

10. Automated scores point towards scalable opportunities for moderation and UX…

11. But automation also raises questions about over-generalization across contexts, and algorithmic transparency

12. Questions? Contact Nick Diakopoulos University of Maryland, College of Journalism Twitter: @ndiakopoulos Email: nad@umd.edu Web: http://www.nickdiakopoulos.com More Info N. Diakopoulos. The Editor’s Eye: Curation and Comment Relevance on the New York Times. Proc. CSCW. March, 2015.