Social Role-Aware Emotion Contagion in Image Social Networks Yang Yang, Jia Jia, Boya Wu, and Jie Tang Department of Computer Science and Technology Tsinghua University As the title suggests, we take image social networks as the basis of our study

Image social network (e. g Image social network (e.g., Flickr) users post photos, which express their emotional statuses. Taking Flickr as an example, users are allowed to publish photos. Comparing with text, image is a more subjective media, which to some extent express users’ emotional status The Spiderman, for instance, must be very happy when taking this photo.

Does Emotion contagion exist in image social networks? Users are connected … Does Emotion contagion exist in image social networks? At the meanwhile, users are connected with each other in image social networks. Thus when Spiderman publishes the photo, it will be exposed to his neighbors and may influence them to publish photos to express different emotions, which in turn influence their own friends, and the influence may further spread. This process is called as emotion contagion, which describes how users influence each other’s emotional statuses and how the influence propagates through the network. A natural question arise here is: does emotion contagion exist in image social networks? #Emotion contagion is a process in which a person or group influences the emotions or behavior of another person or group. Emotion Contagion: The cascade of users’ emotional statuses influence each other

Social Roles of Users Bad boys League of heroes Before we answer the question, we first take a look at the underlying social network where the emotional contagion may occur. We see that the network actually consists of two different communities. One is the league of super heroes, and another one is full of bad boys. There are two types of users worth to mention. One is opinion leaders, who take the central position in each community. For example, Spiderman, who connects other heroes in the league, is definitely an opinion leader. Opinion leaders: users taking central positions in communities

Will social roles influence emotion contagions? Social Roles of Users Bad boys Will social roles influence emotion contagions? League of heroes Another kind of users we are interested in are structural hole spanners, who bridge otherwise disconnected communities, like Mr. Simpsons, who connects heroes and bad boys. So we call these two kinds of users as playing different social roles and the second question we aim to answer in this work is, if the emotion contagion exists, will users with different social roles have different behavior patterns during emotion contagion, in other words, will social roles influence emotion contagions？ Structural hole spanners: users bridge otherwise disconnected communities

Predicting Users’ Emotional Status Input: An image social network G=<V, M, E, R>, where V is a set of users, M is a set of images, E represents following relationships between users, and each element in R (v, m, t) denotes that user v publishes image m at time t. We use a matrix Y to denote users’ emotional status, where yvt indicates v’s emotion at time t. yvt {happiness, surprise, anger, disgust, fear, sadness} Task: Given G, Y, a time stamp t, our goal is to learn Our ultimate goal in this work is to predict users’ emotional status. So we are given a image social network, which consists of users, images published by these users, following relations between users, and the publishing relations between users and images. We use a matrix Y to denote users’ emotional status. Specifically, y_{vt} indicates user v’ emotion at time t. And we consider six emotions: happiness, surprise, anger, disgust, fear, and sadness. Our task is, given the image social network and users’ emotions before time t, we aim to predict users’ emotion at time t [1] Yang, Y.; Jia, J.; Zhang, S.; Wu, B.; Chen, Q.; Li, J.; Xing, C.; and Tang, J. 2014. How do your friends on social media disclose your emotions? In AAAI’14, 2014.

Treat each individual independently Related Work Predicting users’ emotions by jointly modeling images and comments. Yang, Y.; Jia, J.; Zhang, S.; Wu, B.; Chen, Q.; Li, J.; Xing, C.; and Tang, J. 2014. How do your friends on social media disclose your emotions? In AAAI’14, 2014. Predicting users’ emotions in mobile network by considering calling/messaging logs. Tang, J.; Zhang, Y.; Sun, J.; Rao, J.; Yu, W.; Chen, Y.; and Fong, A. Quantitative study of individual emotional states in social networks. TAC’12, 2012. Images drive event engagement (e.g., clicking “like” or adding comments) 100 times faster than text on Facebook. Wang, X.; Jia, J.; Cai, L.; and Tang, J. Modeling emotion influence from images in social networks. IEEE TAFFECT COMPUT’15, 2015. How to better predict users’ emotions by considering emotion contagions? However, these methods treat each individual independently and ignore the correlations/influence among them. Treat each individual independently

Three Qs to Answer Q1: Does emotion contagion exist in image social networks? Q2: Will social roles influence emotion contagion? Q3: How to better predict the emotional status of users in social networks by considering emotion contagion? We conduct several experiments on a Flickr dataset to answer these three questions one by one.

Q1: Existence Q1.1: When your friends are happy, will you be happy? We answer the first question from two aspects. Taking the happiness emotion as an example, we study when my friends are happy, whether I tend to be happy. The x-axis in the figure indicates the number of friends being happy, and the y-axis indicates the probability that I will be happy. We see that when the number of a user’s friends being happy increases, the probability rows roughly linear, which implies that the emotional status of a user will be influenced by her friends.

Q1: Existence Influence Q1.2: When predicting a user’s emotional status, will her friends help? Historical post logs + Previous emotion Image features User v’s emotional status at time t Predict happiness, surprise, anger, disgust, fear, sadness From another angle, we study when predicting a user’s emotions, whether the emotional status of her friends will help. Here we compare two different methods, the first one only considers individual features, such as the users’ historical post log, her previous emotions, visual features of images she publishes, and use these features to feed to a machine learning classifier to predict a user’s emotional status. The second method considers all features the first method includes, plus friends’ emotions as a long vector. + Friends’ emotions

Emotion contagion does occur in image social networks Q1: Existence Influence Q1.2: When predicting a user’s emotional status, will her friends help? Historical post logs + Previous emotion Image features Emotion contagion does occur in image social networks For the results, we see that in most cases, friends’ emotions improve the performance, indicates the friends do help in the prediction. However, exceptions happen in predicting emotions of disgust and fear, which suggests the way we model friend’s emotions here is not precise enough. We will try to solve this issue latter, but for now, I think we are safe to claim that emotion contagion does occur in image social networks. + Friends’ emotions

Still holds in emotion contagion? Q2: Social Role Opinion leaders: 20% of users with largest PageRank scores; Structural hole spanners: 20% of users with lowest network constraint scores; Others are remaining as ordinary users. OL and SH are more influential than ordinary users in information diffusion [Yang’15]. Still holds in emotion contagion? [1] Y. Yang, J. Tang, C. W.-k. Leung, Y. Sun, Q. Chen, J. Li, and Q. Yang. Rain: Social role-aware information diffusion. In AAAI’15, 2015.

Q2: Social Role Happy Fear Happy Fear X: number of friends with different social roles. Y: probability being a certain emotion. Happy Fear

positive emotion delights friends Q2: Social Role Happy Fear X: number of friends with different social roles. Y: probability being a certain emotion. Happy positive emotion delights friends Fear positive emotion tends to delight friends, making them be more happy and less fear

Q2: Social Role Happy Fear Happy Fear X: number of friends with different social roles. Y: probability being a certain emotion. Happy Fear when there are 1-2 users being negative emotion, their common friends tend to be less happy and more fear

“Emotional comfort” phenomena Q2: Social Role Happy Fear X: number of friends with different social roles. Y: probability being a certain emotion. Happy Fear “Emotional comfort” phenomena however, when this number increases, the user will be more happy and less fear “emotional comfort”: when a user is surrounded by a few friends with negative emotions, the user and her friends will comfort each other and get better mood.

Q2: Social Role Happy Fear Happy X: number of friends with different social roles. Y: probability being a certain emotion. Happy Opinion leaders are more influential on positive emotions Fear We also see that opinion leaders are most influential when they are happy, while ordinary users have more influence when they have negative emotional status (e.g., fear). Intuitively, opinion leaders and structural hole spanners are not necessarily as close to their friends as ordinary users. Ordinary users are more influential on negative emotions

Q2: Social Role Happy Fear Happy X: number of friends with different social roles. Y: probability being a certain emotion. Happy Influence of opinion leaders and structural holes change faster than ordinary users. Fear Secondly, we observe that, as the number of infected friends grows, infection probabilities of opinion leaders and structural hole spanners change faster than ordinary users.

Q3: Model P(Y|G): Conditional probability of users’ emotional status given input data

Q3: Model P(Y|G)=πg(.) … g(xvt, yvt): Correlation between v’s emotion and the image she posts at t.

h(yut-t’, yvt): Correlation between v’s emotion at time t and t-t’. Q3: Model P(Y|G)=π{g(.)h(.)} … h(yut-t’, yvt): Correlation between v’s emotion at time t and t-t’.

Q3: Model P(Y|G)=π{g(.)h(.)l(.)} l(yut-1, yvt): How v’s emotion at t is influenced by her friend u’s emotion at t-1. Social role sensitive parameter

Experimental Results Flickr dataset: 2,060,353 images, 1,255,478 users ground truth obtained by user tags Distribution of users’ emotional statuses on Flickr: happiness: 46.2% surprise: 9.7% anger: 8.0% disgust: 5.3% fear:17.3% sadness: 13.5%

Experimental Results Baselines Methods do not consider emotion contagion: SVM, Logistic Regression (LR), Naïve Bayes (NB), Bayesian Network (BN), Gaussian Radial Basis Function Neural Network (RBF). Methods ignore social role information: CRF Our model: Role-aware

Experimental Results Evaluation Metrics: Precision Recall F1 Measure

Experimental Results

(d) Structural hole spanners (a) Ground truth (b) Random users We finally use a case study to demonstrate how different social roles behave in our prediction task. The figure shows the emotion distributions and the degree of happiness of users from different countries. Specifically, each figure from left to right shows the real emotion distributions and happiness degrees of all users, randomly sampled users, opinion leaders and structural hole spanners, respectively. We see that in most countries, the happiness degrees of structural hole spanners and opinion leaders’ are higher than those of the real distribution. This can be explained by the fact that structural hole spanners and opinion leaders are public figures on social networks, who attach much importance to their spreading of positive emotions, while ordinary users tend to share their daily lives. (c) Opinion leaders (d) Structural hole spanners

Conclusion We study the interplay between users’ social roles and emotion contagions by answering 3 questions. Does emotion contagion exist? How social roles influence emotion contagion? How to better predict users’ emotional status? We propose the social role-aware contagion model and validate it on a real social network.

Social Role-Aware Emotion Contagion in Image Social Networks Thank you! Social Role-Aware Emotion Contagion in Image Social Networks Yang Yang, Jia Jia, Boya Wu, and Jie Tang Department of Computer Science and Technology Tsinghua University Contact: SherlockBourne@gmail.com http://yangy.org