1. Predicting Leadership Roles in Email Workgroups Vitor R. Carvalho, Wen Wu and William W. Cohen Carnegie Mellon University CEAS-2007, Aug 2 nd 2007

2. Motivation  Link to leadership studies in hci  Importance of email  Leadership at a distance, communication media, etc.[Butler et al, 2002; Fussell et al, 2001]

3. Overview: Corpus Study  Study: How well we can predict leadership  Textual and “network” features What are the most predictive features  Large and special collection of emails CSPACE corpus (aka GSIA corpus) Leaders (presidents of workgroups) were previously determined

4. CSPACE corpus  15,000 messages from 277 students  Emails associated with a semester-long project (14 weeks) of Carnegie Mellon MBA students  To simulate companies competing for market share and profit, students were divided in 50 companies/teams (4 to 6 students/team)  Very real: student grades largely based (70%) on company financial performance and external board review  Most communication happened inside group  Very rich in task negotiation.  Presidents assigned in the beginning of the game.  Presidents selected other team members through a round-robin draft.

5. Email dataset: network = President … Team A Team B Team CTeam D

6. Evidence from email header: network features  Broadcast messages Sent to all other team members  Non-Broadcast messages Not sent all team members

7. Evidence from email header: network features

8. Textual Features: Email Acts  Cohen et al., EMNLP-04: classification of email content in terms of having “email speech acts”  Examples: Deliver, Request, Commit, Propose, Meeting, etc.  Ciranda: Java toolkit available online

9. Textual Features: Email Acts We also used the associated ranking features, i.e., first (_1), second (_2), last (_last) and one but last (_butlast) 96 features total

10. Experiments 1 10-fold cross-validation using SVM with linear kernel

11. Experiments 2:  Restricting to a single president per group, using All Features: 96% of accuracy F1-measure of 0.882. It correctly predicts the president in 30 out of 34 groups (minimum of 20 messages)

12. Analysis: Feature Selection with  2 test

14. Analysis  Overall, results are interesting Suggests some types of evidence are correlated with leadership  How “natural” is this dataset? Semester long, 14 weeks No language restriction (flames, arguments, cheering, arrangements, gossip, etc.) Grade depended on the team financial performance  Leadership & the choice of leader Laboratory setting of work mitigates the reliability of the conclusions  How much self-selection is going on in choosing "President" of company? Are students behaving according to their expectations of how executives behave? Presidents were selected on a popular vote: the class as a whole (277 students) elected the 50 presidents.

15. Related Work  Community structure and leadership roles in email archives of organizations. [Leuski, 2004][Tyler et al, 2003.]  Very different datasets, methods, validations, etc.

16. Thank you.