1. Let’s Play Poker: Effort and Software Security Risk Estimation in Software Engineering Laurie Williams williams@csc.ncsu.edu 1 Picture from http://www.thevelvetstore.com

2. Another vote for… “Everything should be made as simple as possible, but not simpler.” --Albert Einstein http://imagecache2.allposters.com/images/pic/CMA G/956-037~Albert-Einstein-Posters.jpg

3. Estimation Pictures from http://www.doolwind.com, http://news.cnet.com and http://www.itsablackthang.com/images/Art-Sports/irving-sinclair-the-poker- game.jpg How many engineers? How long? What is the security risk? Planning Poker Protection Poker

4. Effort Estimation: Planning Poker Pictures from http://www.doolwind.com, http://www.legendsofamerica.com/photos-oldwest/Faro2-500.jpg How many engineers? How long?

5. Historical Effort Estimation 5 Pictures from http://www.stsc.hill.af.mil/crosstalk/2003/09/0309hirmanpour_f1.gif, http://www.cs.unc.edu/~stotts/145/cocomo4.gif and http://www.timoelliott.com/blog/WindowsLiveWriter/IntestineBasedDecisionMaking_2C89/gut%20f eel_1.png and http://www.isr.uci.edu/icse-06/images/keynotes/Boehm.jpg and http://www.rallydev.com/images/mike_photo_color.jpg Gut feel often based on: Disaggregation Analogy Expert opinion

6. Coming up with the plan 6 Desired Feature s 30 story points 6 iterations 5 story points/ iteration June 10

7. Estimating “dog points” Estimate each of the dogs below in dog points, assigning each dog a minimum of 1 dog point and a maximum of 10 dog points A dog point represents the height of a dog at the shoulder –Labrador retriever –Terrier –Great Dane –Poodle –Dachshund –German shepherd –St. Bernard –Bulldog 7

8. What if? Estimate each of the dogs below in dog points, assigning each dog a minimum of 1 dog point and a maximum of 100 dog points A dog point represents the height of a dog at the shoulder –Labrador retriever –Terrier –Great Dane –Poodle –Dachshund –German shepherd –St. Bernard –Bulldog 8 More or less accurate? Harder or easier? More or less time consuming?

9. Estimating story points Estimate stories relative to each other –Twice as big –Half as big –Almost but not quite as big –A little bit bigger Only values: –0, 1, 2, 3, 5, 8, 13, 20, 40, 100 9 Near term iteration “stories” A few iterations away “epic”

10. Vote based on: Disaggregation Analogy Expert opinion Diversity of opinion is essential!

11. Not working as fast as planned? 11 Desired Features 30 story points 6 iterations 5 story points iteration June 10 3 story points iteration 10 iterations July 8

12. (Subjective) Results of Planning Poker Explicit result (<20%): –Effort Estimate Side effects/implicit results (80%+): –Greater understanding of requirement –Expectation setting –Implementation hints –High level design/architecture discussion –Ownership of estimate

13. Security Risk Estimation: Protection Poker http://news.cnet.com and http://swamptour.net/images/ST7PokerGame1.gif What is the security risk?

14. Software Security Risk Assessment via Protection Poker

15. Computing Security Risk Exposure Traditional Risk Exposure probability of occurrence Ximpact of loss NIST Security Risk Exposure likelihood of threat- source exercising vulnerability Ximpact of adverse event on organization difficulty enumeration of adversary types motivation of adversaries Proposed Security Risk Exposure ease of attackXvalue of asset -To organization -To adversary Value pointsEase points

16. Protection Poker Overview Calibrate value of “assets” Calibrate ease of attack for requirements Compute security risk (value, ease) of each requirement Security risk ranking and discussion “Diversity of ideas is healthy, and it lends a creativity and drive to the security field that we must take advantage of.” -- Gary McGraw Picture from: http://farm1.static.flickr.com/203/488795952_9007f93c71.jpg

17. Informal discussions of: Threat models Misuse cases Diversity of devious, attacker thinking is essential!

18. Memory Jogger

19. Sum of asset value (e.g. one 20 and one 40) Security Risk Assessment Requirement Ease PointsValue PointsSecurity RiskRanking Req 11100 3 Req 25156 Req 35156 Req 42051003 Req 513 1692 Req 6140 5 Req 7406024001

20. Academic Trial 50 students in undergraduate software engineering course 1. Security cannot be obtained through obscurity alone. 2. Never trust your input. 3. Know your system. 4. Know common exploits. 5. Know how to test for vulnerabilities.

21. Industrial Trial Active participation by all on-site team members Requirements revised for added security fortification Cross site scripting vulnerability found on the spot Expressed need for education on cross site scripting Expressed need for governance to prioritize security fortification Increase awareness of necessary security testing

26. (Subjective) Results of Protection Poker Explicit result (<20%): –Relative security risk assessment Side effects/implicit results (80%+): –Greater awareness understanding of security implications of requirement Collaborative threat modeling Collaborative misuse case development –Requirements changed to reduce risk –Allocation of time to build security into new functionality “delivered” at end of iteration (appropriate to relative risk) –Knowledge sharing and transfer of security information

27. http://www.photosofoldamerica.com/webart/large/254.JPG http://www.cardcow.com/images/albert-einstein-at-beach- 1945-celebrities-28954.jpg