The Coles Notes Approach to Effective Network Security Management Reporting Dave Millier

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Overview Brief Intro Some Terms of Reference A foundation for Reporting Who, Why, How? Typical types of Logs Gathering, Normalization, Archiving Key Performance Indicators Compliance Reporting Management Presentation

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Technical Definitions (From the Devil’s Security Dictionary 2.0) Active X A technology for making Web vulnerabilities more engaging and fun. Change Control A carefully defined and measured process of self-delusion.

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) A form of text communication similar to but far rarer than spam. Single Sign-on A process ensuring that one password gives hackers access to everything. Hash Table The place you roll a joint. Keystroke Loggers Men who type down trees for a living.

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Risk The unavoidable part of life that CEOs try to ignore, CFOs try to hide, CIOs try to understand and CSOs try to control.

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) From CSO Magazine 2005/2006 Surveys 46% of CISOs spend up to 1/3 of their day reading/analyzing reports generated from their security applications 35% of CIOs indicate network security improvements top to-do lists in % of companies report they don’t have established ROI metrics for security risk management 56% of company boards surveyed rarely/never discuss policies, leaving IT Security Mgmt. to make compliance decisions and ensure adherence Some Interesting Statistics

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Considerations for Reporting to Management Align reports with corporate goals Communicate in their language Report residual risk, if it exists Highlight significant trends and events

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) How and Where Do We Get The Data To Report On?

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416)

Collecting & Storing the Data

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Benefits of a good SIM/SEM Aggregate / Normalize data from unrelated devices into useful info Customized reporting to adhere to specific compliance requirements Analyze/correlate information from various devices to identify attacks as quickly as possible Provide the ability to conduct forensic investigations against all data gathered Increase value of existing security devices Improve effectiveness/responsiveness of existing personnel

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Overall Goals of Log Gathering Centralize log data –Fast forensic searching –Contextual reporting –Significant reduction in troubleshooting time –Regulatory/compliance drivers Normalize data –Make all data look the same regardless of source from a searching perspective –Ease of searching Archive data –Regulatory / compliance –Assist with historical investigations –Demonstrate due diligence

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Auditors want to see that there is a PROCESS in place Centralize Logs / Archive Logs Report on unusual activity Identify Action Items for follow-up Document ALL of these steps!!!!! Auditors care more about consistent process than the results themselves Process, Process, Process

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Using the Data for something meaningful: KPIs and Compliance Reporting

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Key Performance Indicators (KPIs) Specific measurement of an organization’s performance in some area of its business One purpose is to give business decision- makers quantifiable measurements of items it has determined important to its long-term success In order to be useful, they must be consistent, have a direct correlation to the area of the business being monitored, and not be susceptible to false readings

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Many Uses of KPIs Planning, Control, Evaluation Managing change Communication Measurement and improvement Resource Allocation Measurement & Motivation Long-term Focus

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Components of a good KPI Descriptive Title for KPI Purpose: Rationale underlying measure Relates to: Business objectives related to item being measured Target: Targets are necessary to evaluate level of performance Formula: The way performance is measured affects how people behave. The right formula ensures the right behaviour Frequency of measurement: Function of the importance of the measure and the volume of data available Frequency of review: Identify how often the gathered measurements should be reviewed Who measures: Identify the person(s) responsible for collecting/reporting the data Source of data: Consistent source of data is vital to performance tracking over time Who owns the measure: who is overall responsible for accuracy, response, etc.? Who acts on the data: person/role should be identified What do they do? Without defined response, measurement is pointless Any notes or comments about the KPI Source: Neely, 1997

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Example of Gathering KPI Information

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Typical Compliance Framework

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Compliance Reporting Focus on providing senior Mgmt. / auditors with current compliance status Unlike KPIs, usually require more detail More focus on identifying anomalies, rather than just reporting on a “number” Should provide access to detailed audit trail for any investigations of events (ticket or case management system)

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Compliance Reporting Goals Provide auditors with a centralized location to perform audit functions against process(es) related to regulatory/compliance requirements Provide attestation to senior C-level executives responsible for the integrity of financial systems, shareholder reporting, etc. Demonstrate “compliance” with established Policies and Enforcement strategies

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Sample Security Reports / KPIs User Identification/Authentication User Account Management User Privileges Configuration Management Security Device Specific Reports (firewall, IDS, spyware, A/V, etc.) Event Activity Monitoring / Logging

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Presenting Results Centralized Repository Display KPIs as both statistical numbers and graphs where possible Granular User-level Access Controls to every report / view Maintain historical copies of pre- generated reports Make sure reports can be saved / printed

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) EXECUTIVE SUMMARY INFORMATION High level summary information for executives and managers. Security Assessments indicate overall security posture, analysis from business and technical perspective, key metrics, case/ticket management. SUB SYSTEM SUMMARY INFORMATION Summary information for security and/or operational sub-systems, with more focused access to information. FORENSICS & REPORTING View correlated logs in near real time. Conduct forensic searches. Generate real-time and historical reports. RAW LOG SOURCES Key devices (firewalls, IDS, network devices, servers, etc.) located throughout the infrastructure. IDEAL REPORTING HIERARCHY

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Sample KPI Reporting

Queen Street East, Suite 200, Toronto, ON, M4L 1H1 - (416) Here’s Gord! Question/Answer At the End of Both Presentations