1. Costing Secure Systems Workshop Report
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Costing Secure Systems Workshop Report
Edward Colbert
Danni Wu
{ecolbert,
21st International Forum on COCOMO & Software Cost Modeling
© USC-CSE
19 November 2018
© USC-CSE

2. In Case You Aren’t Sure That Security Is Important
© USC-CSE
19 November 2018

3. Workshop Participants
Ed Colbert, USC, Moderator
Danni Wu, Scribe
Don Reifer, Reifer Inc.
Martha Leonette
Anca-Jiliana Stoica
Rita Creel
Ron Owens
Barry Boehm
© USC-CSE
19 November 2018

4. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Goal Of Workshop
Review Research
Draft model for early costing of system security
Extensions to COCOMO II for development of secure software systems (“COSECMO”)
Gather expert opinion
Invite Data
MetaH provides semantics & supporting tools
UML provides graphic front-end
© USC-CSE
19 November 2018
© USC-CSE

5. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Workshop Agenda
Introduction
Review Early Estimation Model
Review COCOMO Security Extension ("COSECMO")
Delphi
© USC-CSE
19 November 2018
© USC-CSE

6. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Outline
Project Schedule
Early Estimation Model
COCOMO Security Extension ("COSECMO")
To Do
© USC-CSE
19 November 2018
© USC-CSE

7. Cost Model for System Security Increment 1 (Feb – July ’04)
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Cost Model for System Security Increment 1 (Feb – July ’04)
Task Element
Activities
1. Develop Early Estimation Model
Prototype model
2. Sources of Cost
Identify, define, scope sources of cost
Relate sources of cost to FAA WBS
Recommend type of CER for each
3. Secure Product Taxonomy
Identify, define, scope product elements
4. COCOMO II Security Extensions
Refine model form and data definitions
5. COCOTS Security Extensions
Explore security aspects in COCOTS data collection
© USC-CSE
19 November 2018
© USC-CSE

8. Cost Model for System Security Increment 2 (Aug ’04 – Mar ’06)
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Cost Model for System Security Increment 2 (Aug ’04 – Mar ’06)
Task Element
Activities
1. Develop Early Estimation Model
Experimental use & refinement
2. Sources of Cost
Prioritize sources of cost needing CER’s
Refine, prototype, experiment with top-priority CER’s
Relate to scope of COCOMO II security extensions
3. Secure Product Taxonomy
Experimental use, feedback, and refinement
4. COCOMO II Security Extensions
Refine, scope, form, definitions based on results of Tasks 1-3
Experimentally apply to pilot projects, obtain usage feedback
5. COCOTS Security Extensions
Develop initial scope, form, definitions based on results of Tasks 1-4
© USC-CSE
19 November 2018
© USC-CSE

9. Cost Model for System Security Increment 3 (Mar ’06 – Feb ’07)
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
We are in middle of inc.
Cost Model for System Security Increment 3 (Mar ’06 – Feb ’07)
Task Element
Activities
1. Develop Early Estimation Model
Evolution; integration with other models
2. Sources of Cost
Refine sources of cost, CER’s based on usage feedback
Integrate with other models
Address lower-priority CER’s as appropriate
3. Secure Product Taxonomy
Monitor evolution
4. COCOMO II Security Extensions
Baseline model definitions
Collect project data
Develop initially calibrated model; experiment and refine
© USC-CSE
19 November 2018
© USC-CSE

10. Cost Model for System Security Increment 4 (Apr ’07 – Mar ’08)
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Cost Model for System Security Increment 4 (Apr ’07 – Mar ’08)
Task Element
Activities
1. Develop Early Estimation Model
Evolution
Integration with other models
2. Sources of Cost
Refine sources of cost, CER’s based on usage feedback
Integrate with other models
3. Secure Product Taxonomy
Monitor evolution
4. COCOMO II Security Extensions
Collect project data
Develop initially calibrated model
Experiment & refine
© USC-CSE
19 November 2018
© USC-CSE

11. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Outline
Project Schedule
Early Estimation Model
COCOMO Security Extension (“COSECMO")
Data Mining
To Do
© USC-CSE
19 November 2018
© USC-CSE

12. Formula for Cost of System & Security
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Formula for Cost of System & Security
Ctotal = CInitial/Mission Analysis + CInvestment Analysis + CSystem Engineering + CDev & Imp + CSys of Sys Integration + CInstall/Deployment + CO&M + CDisposal
CDev & Imp = CDesign & Build HW + CDesign & Build SW + CPurchased Services + CCOTS-Sys + CEnv-Mods-design + CBus-Proc-Re-engineering
Ctotal (Security) = Ctotal (with security) – Ctotal (without security)
COTSYS  Commercial of the Shelf Systems
O&M  operation & maintenance
Env-Mods-Design  Design of Modifications to environment that needs to be implemented during installation/deployment (e.g. add steal-re-enforced cement barriers)
Bus-Process Re-engineering  Re-engineering/Design of business processes that needs to be implemented during installation/deployment, operation & maintenance, or disposal
C = Cost
© USC-CSE
19 November 2018
© USC-CSE

13. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
5th Prototype Tool Screenshot#4 Advance Estimate — Cost Item by Parametric Models
© USC-CSE
19 November 2018
© USC-CSE

14. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Outline
Project Schedule
Early Estimation Model
COCOMO Security Extension (“COSECMO")
Data Mining
To Do
© USC-CSE
19 November 2018
© USC-CSE

15. Effect Of Security On COCOMO II
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Effect Of Security On COCOMO II
Source lines of code (SLOC’s) increased
Implementation of Security Functional Requirements (SFR’s)
Effort to produce code increased by
Security Assurance Requirements (SAR’s)
A few Security Functional Requirements (SFR’S)
Effort for “outer phases” of life–cycle (e.g. Inception, Transition) increased by
Additional documents
Additional activities
e.g. definition of security roles, certification
© USC-CSE
19 November 2018
© USC-CSE

16. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Effect Of Security Functional Requirements On SLOC’s & Computed Effort (cont.)
Computation of total effort
PMtotal = PMTSF + PMapplication PMCertification/Validation/Accreditation
TSF often developed at higher level of security
© USC-CSE
19 November 2018
© USC-CSE

17. New Security Driver (SECU) (cont.)
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
New Security Driver (SECU) (cont.)
6 COCOMO levels ≈ 7 CC EAL’s (or equivalent activity)
Treating EAL 1 as Nominal & EAL 2 as Nominal+50 (or High-50 )
Tailoring/Modification/Addition of SAR’s handled by increasing/decreasing base level
Rating Level
Estimated Scale Value
Rating Scale (Refer to Supplement for details)
Nominal (NOM)
0.00
No security requirements of added protection
High (HI)
1.0
Informal security requirements, methodically tested and checked
Very High (VH)
1.5
Methodically designed, tested and checked
Extra High (XH)
2.0
Semi-formally designed and tested
Super High (XXH)
5.0
Semi-formally verified designed and tested
Ultra High (XXXH)
10.0
Formally verified designed and tested
© USC-CSE
19 November 2018
© USC-CSE

18. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
COSECMO Estimation Trends Effort for Different Size Projects at Assurance Levels
Plot of projects where only SECU & effort increasing drivers
Efforts seem a little low based on values from Orange Book projects
© USC-CSE
19 November 2018
© USC-CSE

19. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
COSECMO Estimation Trends Effort by Assurance Levels for Different Size Projects
Plot of projects where only SECU & effort increasing drivers
Efforts seem a little low based on values from Orange Book projects
© USC-CSE
19 November 2018
© USC-CSE

20. Barry’s Advice Follow KISS principle
Keep It Simple, Stupid
Or as Einstein said
“Keep it as simple as possible, but no simpler”
© USC-CSE
19 November 2018

21. Proposed Changes to COSECMO
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Proposed Changes to COSECMO
Reviewing decision to make SECU driver a scale factor
Calibration issues
Reduce model complexity by
Eliminating guide for other drivers &
Re-integrate effect into SECU
Discuss with customer simple ways to estimate Certification, Validation, Accreditation
Eliminate “Validation”
Define as percent of development cost?
© USC-CSE
19 November 2018
© USC-CSE

22. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Outline
Project Schedule
Early Estimation Model
COCOMO Security Extension (“COSECMO")
Data Mining
To Do
© USC-CSE
19 November 2018
© USC-CSE

23. Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
To Do
Refine costing prototypes
Get more feedback from security community
Refine models
Refine Delphi
Collect & analyze data
We need data!
Write papers & Ph.D. thesis (theses?)
© USC-CSE
19 November 2018
© USC-CSE

24. Next Costing Secure Systems Workshop
Costing Secure Systems Update, USC-CSE 20th Annual COCOMO/SCF Forum
11/19/2018
Next Costing Secure Systems Workshop
Date: 14 February 2007
Time: 8AM –5PM
Location: USC’s CSSE
Part of CSSE’s Annual Research Review & Executive Forum
See csse.usc.edu & click on Events
Cost:
Workshop is free
© USC-CSE
19 November 2018
© USC-CSE