Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Adaptive Metrics Develop metrics that determine how well we are adapting to our ever-changing environment. Fitness Functions Identify dependencies and requirements for optimum productivity around the Laboratory. Measure the impact of a localized failure of one entity across the entire organization. Balanced Score Card Review our program from a balanced perspective. Provide metrics by which we can manage. Optimizing Performance Management Gina Fisk, LANL Senior Cyber Security Manager StrategyBudgetingInitiativesMeasures

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Starting Point – Remove the Clutter Remove metrics that we can’t use to manage our information security program. How many customers called our help desk. How many connections were deflected by our firewall. How many times our network was scanned, etc. Bin the remaining metrics into the BSC framework for a Phase I BSC. Financial. Customer. Internal Processes. Learning and Growth. 2

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Determine Impacts of Failure Conduct IT Impact Analysis Determine the cost to an organization if various IT services failed for variable lengths of time. Network, , local storage, etc. Calculate Impact Rating for each IT Service. 1/n, where n is the average number of days until an organization has lost 100% of productivity. Calculate the Daily Monetary Impact of the Loss of that IT Service for an organization. Calculate the overall productivity cost for the Laboratory as a whole based on that loss. 3

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Focus Areas From IT Impact Analysis results, identify IT Services with largest impacts to productivity. Loss of Accreditation of systems Loss of local network access Loss of Loss of Oracle Loss of Internet access Goals that the CIO and CISO had set for the organization in the Strategy Map. Develop metrics based on these focus areas and develop Phase II of the BSC. 4

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Information Security Strategy Map 5 INTERNAL AND PROCESS PERSPECTIVE LEARNING AND GROWTH PERSPECTIVE LG1. Attract, develop, and retain highly skilled security professionals LG2. Develop risk-focused and customer-centric culture LG3. Align employee training with strategic initiatives CUSTOMER PERSPECTIVE Competency Contribution IP1. Streamline compliance program to achieve 100% of scheduled accreditations IP2. Optimize operations to reduce KTLO by 10% per BU IP3. Enhance performance through implementation and management of service agreements Achieve Operational Excellence IP4. Mature IT governance processes and increase partner participation IP5. Build a structured, transparent and collaborative regulator relationship IP6. Promote transparency and performance through holistic metrics program Create and Support Internal Programs and External Partners IP7. Propose and deliver business-enabling information security solutions IP8. Mature IT risk program to drive security, portfolio, and governance decisions IP9. Enhance red network monitoring and vulnerability management Deliver Innovative Security Solutions FINANCIAL PERSPECTIVE Competitive Advantage Operational Excellence C1. “Understand and consistently deliver what I need” C2. “Keep me out of security and compliance trouble” C3. “Establish a positive reputation which will help me with my customers” C4. “Become a trusted partner by helping me solve my challenging problems” F2. Maximize operational efficiency F4. Facilitating acquisition of new business through best-in-class IT security execution F3. Minimize IT enterprise risk Maximize mission enablement by balancing risk and value (F1)

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Balanced Score Card Financial (F1-F4) Security unit costs On-time rate of accreditations Enterprise risk rating Business impact of incidents Projects on- time/budget Cyber PBI ratings Lower unit costs 100% on time Maintain.3 rating <25hrs/Q<10% variance>95% green Target Initiative Customer (C1-C4) CommunicationCompliance Customer Support Program Input Time per accreditation Customer Satisfaction >80% survey scores >70% survey scores >80% survey scores >90% governance participation >95% CA/avg times >80% survey scores Target Initiative Internal Processes (IP1-IP7) AOE: Opex reduction AOE: SLA performance CSIPP: unplanned work DISS: AOP risk mapping DISS: BP tied to risk DISS: Red capabilities >=2.5% Q/Q<10% variance<=3/Q>=80% >=30% key processes Positive trend Target Initiative Hits target. Initiative on track Short of target. Initiative recoverable Failed process. Initiative not recoverable Target not defined. No initiative Learning and Growth (LG1-LG3) Training roadmap Planned role rotations Attrition reduction Strategic training XX <10% schedule variance >=1/Q Reduced attrition rate >50% training mapped to initiatives XX Target Initiative Note: BSC target performance scores are represented here for explanatory purposes only 6

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Fitness Functions Fitness functions measure the overall health of an organization by measuring not only performance, but also the performance of those organizations on which we are dependent to achieve our goals. If the performance of one of the dependencies fails, there are ramifications throughout the entire organization. Using the fitness scores of dependent organizations, we can measure the impact of a localized failure of one entity across the entire organization, providing valuable measurements of the actual cost of security incidents, network outages, etc. We can trend these scores to evaluate performance at various levels of the organization. System Administration System Administration Network Services Network Services Identity Management Identity Management Scientific Computing Core Services Production Cycles Visualization Services Backups and Storage Security Infrastructure Science and Engineering C&APhysical Infrastructure PublicationsContractsPatents Dependencies Reliant Organizations 7

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Example Fitness Function Framework Fiscal Responsibility (weight: 20%) Milestones and deliverables (quality, timeliness) Expenditures (percentage over budget) Customer Productivity (weight: 15%) Services maximize productivity around organization (uptime, etc) Customer Orientation (weight: 15%) Responsiveness to the customer (SLAs, etc) Improving Security (weight: 15%) Progress made toward improving security against our current threat environment (hardening tools, etc) Institutional Responsibilities (Weight: 20%) PBI deliverables and reporting (quality, timeliness) CAP deliverables and reporting (quality, timeliness) Metrics reporting (quality, accuracy) Goal-Based Initiatives (weight: 10%) Progress made against organizational goals. 8

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Fitness Function Example Fiscal Responsibility (weight: 20%).89 Timeliness of deliverables and milestones:.83 % of projects +/- 10% of budget allocation:.95 Laboratory Productivity (weight: 15%).98 Uptime of service:.98 Customer Orientation (weight: 15%).89 Customer Satisfaction Rating.89 Improving Security (weight: 15%) **.56** Progress made toward improving security against our current threat environment (hardening tools, etc).56 Institutional Responsibilities (Weight: 25%).68 PBI deliverables and reporting (quality, timeliness).90 CAP deliverables and reporting (quality, timeliness).75 Metrics reporting (quality, accuracy).40 Goal-Based Initiatives (weight: 10%).98 Progress made against initiatives..98 FITNESS SCORE: Note: Fitness scores are represented here for explanatory purposes only

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Fitness Score Trends Fitness scores allow us to watch for trends and to manage by our metrics. See how major changes affect our performance from month to month. Change in Management Change of Platform Change of Vendor, etc. 10

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Adaptive Metrics IT Impact Analysis provide us with costs of the failures of IT Services. We have the data on our ever changing threat environment. The fitness functions allow us include “moving target” metrics, which change each month, to measure our performance against our current threat environment. 11

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D **Adaptive Metric Example “Improving Security (weight: 15%)” Identify top threats for the month. Phishing, Windows vulnerability, Oracle vulnerability. Calculate cost of failure of these services across the organization per month. $200K, Windows: $500K, Oracle: 800K Overall Budget: 10% in jeopardy Review % of security effort we are placing on these areas ($$$ spent). 5%, Windows: 31%, Oracle: 20% Weight the fitness function by how responsive we are to these areas. 56% of our budget is spent on our top threat areas. 12 Note: Threats and budgets represented here for explanatory purposes only

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Managing by the Metrics Our budget, metrics, and initiatives are actionable and directly tied to our goals. Our use of the Balanced Score Card helps us ensure uniform management of our business. Our use of the Fitness Functions help us trend our metrics effectively and monitor the major changes. We can trend our components individually or as a whole, organizationally or institutionally. Our use of Adaptive Metrics keep our outlook fresh and defendable. 13