Metrics for Organizational Cybersecurity Practices Benjamin C. Dean Consultant to OECD Secretariat Metricon X Stevens Institute of Technology Hoboken, NJ, USA March 22, 2019

Agenda The problem OECD project overview Framework Lessons + recommendations Q&A Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech

The Problem: unanswerable questions Cybercrimes against businesses, US Dept. of Justice, 2005

The Problem: laundry lists ABACUS survey, Australia, 2009

The Problem: not technically informed Survey on information security in businesses, Korea, 2015 Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech

The Problem: poorly worded concepts Community survey on ICT usage and e-commerce in enterprises, Eurostat, 2019 Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech

Project overview Goals: Establish a measurement framework of digital security risk management practices Make it: Conceptually clear Succinct Organisational – not technical Focus on what is done i.e. practices Relevant to policymakers Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech

Project overview Timeline: 2-year project Audience: policymakers, national statistical offices, insurers Final report (soon to be published): Section 1: methodological issues Section 2: the measurement framework Section 3: pilot results Recommendations Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech

Framework built on OECD principles

Cognitive testing & Pilot Survey instrument with six modules & eighteen indicators Uses OECD “model survey” framework Cognitive testing in Brazil: Jan – Apr 2018 Pilot testing with FERMA: Jun – Sept 2018 Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech

Industries to which the respondent enterprises belong A DEMOGRAPHICS A1 Geographic location A2 Size A3 Economic activity A4 Turnover A5 Digital intensity Size of respondent enterprises, by headcount Size class Number of responses Percentage of total Under 10 1 10 to 49 3 4 50 to 249 5 250 to 499 6 500 to 999 8 1000 to 2499 32 40 2500 to 4999 5000 to 9999 10000 or more 18 23 Total 80 Industries to which the respondent enterprises belong Manufacturing: 24% Financial and insurance: 16% Transportation & storage: 13%

Who is in charge of managing digital security risk of the enterprise? B DIGITAL SECURITY RISK GOVERNANCE B1 Responsibilities for digital security risk allocated to a specific role within the organisation B2 Policy in place to manage digital security risk B3 Process in place to monitor and review digital security risk management B4 Structures or processes in place to enable cooperation and for reporting on digital security risk management within the enterprise Who is in charge of managing digital security risk of the enterprise? N = 80

C DIGITAL SECURITY RISK ASSESSMENT PRACTICES C1 Assess digital security risk as part of the overall enterprise risk management C2 Regularly take specific actions as part of the digital security risk assessment Who carries out the following activities as part of digital security risk assessment for your enterprise? N = 80

D DIGITAL SECURITY RISK REDUCTION PRACTICES D1 Took risk reduction measures D2 Share information on threats, vulnerability, incidents and risk management practices or security measures Do you share information on digital security threats, vulnerability, incidents and risk management practices or security measures? N = 80

E DIGITAL SECURITY RISK TRANSFER PRACTICES E1 Use insurance to transfer digital security risk E2 Did not purchase an insurance policy, by reason for non-adoption E3 Transfer digital security risks through an insurance policy, by type of risks transferred E4 Adopt other risk transfer practices Which of the following risks are covered through your insurance policy/policies? N = 44

F DIGITAL SECURITY RISK MANAGEMENT AWARENESS AND TRAINING F1 Adopted awareness-raising and training practices on digital security risk management Over the past year did your enterprise perform any of the following practices? N = 80

Lessons learned + recommendations Cognitive testing and pilot yielded insights Further reduce number of indicators Simplify language Move toward maturity model Better assess the ‘depth’ of practices Metricon X 22 March 2019, New Jersey, USA Benjamin C. Dean @benjamindean | ben@iconoclast.tech