1. Metrics for Organizational Cybersecurity Practices
Benjamin C. Dean
Consultant to OECD Secretariat
Metricon X
Stevens Institute of Technology
Hoboken, NJ, USA
March 22, 2019

2. Agenda The problem OECD project overview Framework
Lessons + recommendations
Q&A
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |

3. The Problem: unanswerable questions
Cybercrimes against businesses, US Dept. of Justice, 2005

4. The Problem: laundry lists
ABACUS survey, Australia, 2009

5. The Problem: not technically informed
Survey on information security in businesses, Korea, 2015
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |

6. The Problem: poorly worded concepts
Community survey on ICT usage and e-commerce in enterprises, Eurostat, 2019
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |

7. Project overview
Goals: Establish a measurement framework of digital security risk management practices
Make it:
Conceptually clear
Succinct
Organisational – not technical
Focus on what is done i.e. practices
Relevant to policymakers
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |

8. Project overview Timeline: 2-year project
Audience: policymakers, national statistical offices, insurers
Final report (soon to be published):
Section 1: methodological issues
Section 2: the measurement framework
Section 3: pilot results
Recommendations
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |

9. Framework built on OECD principles

10. Cognitive testing & Pilot
Survey instrument with six modules & eighteen indicators
Uses OECD “model survey” framework
Cognitive testing in Brazil: Jan – Apr 2018
Pilot testing with FERMA: Jun – Sept 2018
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |

11. Industries to which the respondent enterprises belongA
DEMOGRAPHICS A1
Geographic location A2
Size A3
Economic activity A4
Turnover A5
Digital intensity
Size of respondent enterprises, by headcount
Size class
Number of responses
Percentage of total
Under 10 1
10 to 49 3 4
50 to 249 5
250 to 499 6
500 to 999 8
1000 to 2499 32 40
2500 to 4999
5000 to 9999
10000 or more 18 23
Total 80
Industries to which the respondent enterprises belong
Manufacturing: 24%
Financial and insurance: 16%
Transportation & storage: 13%

12. Who is in charge of managing digital security risk of the enterprise?B
DIGITAL SECURITY RISK GOVERNANCE B1
Responsibilities for digital security risk allocated to a specific role within the organisation B2
Policy in place to manage digital security risk B3
Process in place to monitor and review digital security risk management B4
Structures or processes in place to enable cooperation and for reporting on digital security risk management within the enterprise
Who is in charge of managing digital security risk of the enterprise?
N = 80

13. C
DIGITAL SECURITY RISK ASSESSMENT PRACTICES C1
Assess digital security risk as part of the overall enterprise risk management C2
Regularly take specific actions as part of the digital security risk assessment
Who carries out the following activities as part of digital security risk assessment for your enterprise?
N = 80

14. D
DIGITAL SECURITY RISK REDUCTION PRACTICES D1
Took risk reduction measures D2
Share information on threats, vulnerability, incidents and risk management practices or security measures
Do you share information on digital security threats, vulnerability, incidents and risk management practices or security measures?
N = 80

15. E
DIGITAL SECURITY RISK TRANSFER PRACTICES E1
Use insurance to transfer digital security risk E2
Did not purchase an insurance policy, by reason for non-adoption E3
Transfer digital security risks through an insurance policy, by type of risks transferred E4
Adopt other risk transfer practices
Which of the following risks are covered through your insurance policy/policies?
N = 44

16. F
DIGITAL SECURITY RISK MANAGEMENT AWARENESS AND TRAINING F1
Adopted awareness-raising and training practices on digital security risk management
Over the past year did your enterprise perform any of the following practices?
N = 80

17. Lessons learned + recommendations
Cognitive testing and pilot yielded insights
Further reduce number of indicators
Simplify language
Move toward maturity model
Better assess the ‘depth’ of practices
Metricon X
22 March 2019, New Jersey, USA
Benjamin C. Dean
@benjamindean |