Data Mining of s to Support Periodic & Continuous Assurance Glen L. Gray California State University at Northridge Roger Debreceny University of Hawai`i at Mānoa 5th Symposium on Information Systems Assurance Toronto: October 2007

In this Presentation Continuous monitoring of s – why? Technologies Social Network Analysis Text analysis Challenges Opportunities

Continuous Monitoring of s – Why? Increased focus on forensic approaches to auditing Increased interest in continuous assurance and monitoring of business processes s = Organization’s DNA Evidential matter on: Employee & management fraud (overrides) Compliance (e.g., HIPAA) Loss of intellectual property Corporate policies

Enron Archive Released by Federal Energy Regulatory Commission 500K s 151 Enron employees Cleaned version at Carnegie Mellon Relational DB version at USC eport.pdf eport.pdf

Mining Targets

Content Analysis

Key Word Queries Yes, people do say self-incriminating things in their s Fraud Corporate dysfunction Overwhelming false positives Need “smart” compound queries Good continuous auditing (CA) candidate Already scanning for spam, porn, etc.

Sender Deception -- Content Deceptive s include: Fewer first-person pronouns to dissociate themselves from their own words Fewer exclusive words, such as but and except, to indicate a less complex story More negative emotion words because of the sender’s underlying feeling of guilt More action verbs to, again, indicate a less complex story

Sender Deception -- Identification Writeprint features Lexical -- characters & words Function words Root words Syntactic -- sentences Structural -- paragraphs Content-specific

Sender Deception -- Identification Number of potential features unlimited Optimum number can vary by context and language Developing user profiles and comparing new s to profiles would be challenging for real-time CA

Temporal/Log Analysis

Volume & Velocity Volume = number of s a person sends and/or receives over a period of time. Velocity = how quickly the volume changes. Many external factors (e.g., vacations, seasonal activities, etc.) impact these numbers Need “rolling histogram”

Volume & Velocity Key issue -- determining the optimum time intervals to sample the data Continuous monitoring cannot be continuous in terms of sampling in real time Comparing hourly, daily, and even weekly volumes and velocities will result in many false positives Optimum time internal could vary by job title

Social Network Analysis

Social relationships as an undirected graph Importance of understanding relationships within the flow of exchanges

Social Network Analysis in s s semi-structured data sender primary recipient(s) copied recipient(s) date subject line Social groups and cliques CA = who doesn’t belong?

Thread Analysis – This? Time S R C C SR C C R C C S S R C C

Thread Analysis – Or this? Time S R C C S R R C S C R RS R

Integrating Content Analysis and Social Network Analysis

Challenges of Mining Textual Inconsistent use of abbreviations Misspelled words Smileys etc. etc. Replies, replies, and more replies… Inability to identify: Identities of participants Roles and responsibilities

What Enron s Show? People do say the darnest things What did he know and when did he know it? Verified numerous bodies of data mining research Content analysis Social network analysis

Tools Content monitoring eSoft Corporation’s ThreatWall Symantec’s Mail Security 8x00 Series Vericept Corporation’s Vericept Content 360º Reconnex Corporation’s iGuard Appliance InBoxer, Inc. Anti-Risk Appliance Social networks Microsoft SNARF Heer Vizter

Research Opportunities

Research Questions Role of monitoring in overall CA environment? Join SNA with examination of textual patterns. Link SNA with control environment Frauds/control overrides footprint? What cleaning is required for CA purposes? Privacy and policy issues? Lessons from existing commercial products?

Your Questions Thank You