1. Detecting Changes in Social Networks Using Statistical Process Control Cadet Matthew R. Webb Advisor: Major Ian McCulloh, D/Math USMA 20 May 2007 NetSci 2007 Contributed Talk

2. Agenda Social Network Change Detection –Components Social Network Analysis Statistical Process Control –Feasibility Study Application: Al-Qaeda –Method –Results –Lessons Learned Application: TOEP –Models Future Research and Applications

3. What is Social Network Change Detection? Social Network Analysis –the mathematical methodology of quantifying connections between individuals and groups. Statistical Process Control (SPC) –Calculates a statistic from sequential measurements of a random process and compares it to a control limit. –Used extensively in quality engineering

4. Network Measures Density Betweenness Centrality Closeness Centrality Many More Graph courtesy of Steve Borgatti

5. Statistical Process Control Cumulative-Sum Control Chart –Good at detecting small changes in mean over time –Built-in change point detection –Requires normally distributed data Calculate Z t transform value for each time-period, t. Two Charts (To Detect An Increase or Decrease) Chart Signals when C + or C - statistic exceeds control limit Reference value and control limit are arbitrarily set for each process

6. Feasibility Study Cadet Summer Training –How does communication effect an organization’s performance? –Can changes in communication patterns be detected in larger, well-established organizations? Time Experience

7. Al-Qaeda Application The Office of Naval Research maintains network data on Al-Qaeda. Ran CUSUM Control Chart for three Network Measures (Betweenness, Closeness, and Density)

8. Results Change Point The control chart signals a change in the network in 2001. Most Likely Estimate of the change point is 1997: - Re-establish base in Afghanistan - Terrorist Attack in Luxor, Egypt - Feb ’98 Islamic Front - Embassy bombings in ’98 1997 was a critical planning year for Al-Qaeda Control Limit = 4 Signal

9. Lessons Learned SPC Change Detection Method Can Detect Changes in a Social Network. Further Testing Requirements –Need Dataset with Higher Resolution –More Complete Network Information

10. TOEP Application Tactical Officer Education Program is a master’s degree program for 24 Army officers. Patch on their OUTLOOK allows collection of all sent emails and reconstruction of their email network.

11. Results Monitored Email Traffic for 24 Weeks –Collected viable data from 9 TOEPs –Network measures normally distributed Developed Model of TOEP Closeness

12. Regression Model Used data from first semester to develop model –TOEP planning calendar and interviews to establish possible predictors (Group Projects, Administrative Changes, Academic Requirements, and Planned Social Outings). Closeness Model Conducted SPC on closeness predictions to detect network changes during second semester Closeness = 0.18 - 0.11 Group Projects + 0.11 Social + 0.0074 # of Emails Predictor CoefSE Coef T P VIF Constant 0.180.034 5.40 0.000 Group Projects -0.110.050 -2.13 0.046 1.3 Social 0.110.040 2.89 0.009 1.3 # of Emails 0.00740.00084 8.77 0.000 1.0 S = 0.096 R 2 = 82.4% R 2 (adj) = 79.8% PRESS = 0.30R 2 (pred) = 70.9%

13. Most Likely Estimate of the change point is Week 21: - The week prior to Spring Break - Two weeks prior to Comprehensive Exam - Interviews with TOEPs cite this as the last week for study before the exam Week 21 represents a change in previous group dynamics from first semester SPC on Closeness Change Point Control Limit = 3 Signal

14. Future Research and Applications Further Research –Continue to Monitor TOEP Email Network Compare Data across multiple years Optimize k value and control limit to detect desired type of changes –Future Datasets Use networks with less variance Monitor a “more routine network” to establish better baselines –Examine Possible Relationships between Network Measures and an Organization’s Performance Applications for this Method –Evaluating Friendly Command and Control Networks –Monitoring Terrorist Communication Networks

15. USMA Network Science Workshops USMA/ARI Network Science Workshop 18-20 April 2007 http://www.netscience.usma.edu/nsw/ USMA/ASA-ALT Network Science Workshop 18-20 April 2007

16. Questions?