Proportion of Original Tweets How Does That Make You Feel? Detecting and Understanding Abuse in Context Emily Grace, Ph.D., Dustin Arendt, Ph.D., Svitlana Volkova, Ph.D. OBJECTIVE APPROACH EXPERIMENTAL RESULTS Classification of affect presence or +/- polarity in replies Regression of degree of affect expressed in replies Predictive Tasks LSTMs with pre-trained GloVe embeddings AdaBoost (baseline) Models Number of replies Inverse variance of affect assignments Length of replies Sample Weighting Tweet content, behavior Style, psycholinguistic markers User meta-data Predictive Signals RQ2: What types of emotion are easier to predict? Classification Fear, joy, and anger are easier to predict than disgust, sadness and surprise More extreme levels of sentiment and emotion are more easily distinguished Regression Joy is the easiest to predict Predicting specific level of affect evoked is significantly harder than classifying into high/low emotion levels Twitter Data: 200K tweet-reply pairs in English RQ1 How to identify features of Twitter content that evoke certain sentiments and emotions RQ2 How to detect cases where tweet content is not sufficient to predict reply sentiment RQ3 How to understand behaviors that fall outside of normal social dynamics on social media QUALITATIVE ANALYSIS RQ2: When is the evoked affect not predictable? RQ1 & RQ3: What types of tweets generate positive versus negative replies? RQ1 & RQ3: What features are predictive of different affects? Error Analysis Label noise: incorrect sentiment assignments Outside knowledge: references to external events Media: images or other media posted with tweet Conversational drift: replies introduce new topics Abuse/unrelated-anger: unprovoked angry or abusive responses to innocuous tweet More negative replies More Positive replies Proportion of Original Tweets FUTURE WORK Going beyond text content: use results from error analysis to identify and implement additional predictive features – such as the images or other media content e.g., URLs of the tweets Abuse detection: leverage the model results for the real-time detection of abusive and harassing behavior on social media by flagging model prediction errors