Secure Software Development Abuse Cases Chapter 8 Rasool Jalili & M.S. Dousti Dept. of Computer Engineering Fall 2010

Software Security 2 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Abuse Cases

Software Security 3 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Software development is all about making software do something. The focus is on functionality.  when software vendors sell products, they talk about what their products do to make customers' lives easier, improving business processes,or doing something else positive. UML, use cases, and other modeling and design tools allow software people to formalize what the software will do. Assumption: the system won't be intentionally abused. But what if it is? Example Use Cases in a Payroll system: –The system allows users in the HR management group to view and modify salaries of all employees. –The system will only allow a basic user to view his or her own salary. Software development is all about making software do something. The focus is on functionality.  when software vendors sell products, they talk about what their products do to make customers' lives easier, improving business processes,or doing something else positive. UML, use cases, and other modeling and design tools allow software people to formalize what the software will do. Assumption: the system won't be intentionally abused. But what if it is? Example Use Cases in a Payroll system: –The system allows users in the HR management group to view and modify salaries of all employees. –The system will only allow a basic user to view his or her own salary. Abuse Cases

Software Security 4 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 More experienced consumers may say: "We want the software to be secure" or "We want the software to be reliable." In some cases, these kinds of wants are being formally and legally applied in SLAs and acceptance criteria regarding various system properties. The problem is that security, reliability, and other software -ilities are complicated In order to create secure and reliable software, abnormal behavior must somehow be anticipated. Idea: get out your black hat and think like a bad guy. More experienced consumers may say: "We want the software to be secure" or "We want the software to be reliable." In some cases, these kinds of wants are being formally and legally applied in SLAs and acceptance criteria regarding various system properties. The problem is that security, reliability, and other software -ilities are complicated In order to create secure and reliable software, abnormal behavior must somehow be anticipated. Idea: get out your black hat and think like a bad guy.

Software Security 5 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Risk analysis is a big challenge as you start with a blank page and anticipate things that will go wrong. Same goes for testing (especially adversarial testing and penetration testing). The core of these touchpoints are hypotheses of what might go wrong. That's what abuse cases are all about. Abuse cases (misuse cases) are a tool that can help you begin to think about your software the same way that attackers do. With Qs “What can go wrong here?" or better, "What might some bad person cause to go wrong here?" software practitioners are more likely to uncover exceptional cases and security requirements. Think about what motivates an attacker. Pretend you're the bad guy. Now ask yourself: "What do I want?" Some ideas: –I want to steal all the money. –I want to learn the secret ways of the C-level executives. –I want to be root of my domain. Risk analysis is a big challenge as you start with a blank page and anticipate things that will go wrong. Same goes for testing (especially adversarial testing and penetration testing). The core of these touchpoints are hypotheses of what might go wrong. That's what abuse cases are all about. Abuse cases (misuse cases) are a tool that can help you begin to think about your software the same way that attackers do. With Qs “What can go wrong here?" or better, "What might some bad person cause to go wrong here?" software practitioners are more likely to uncover exceptional cases and security requirements. Think about what motivates an attacker. Pretend you're the bad guy. Now ask yourself: "What do I want?" Some ideas: –I want to steal all the money. –I want to learn the secret ways of the C-level executives. –I want to be root of my domain.

Software Security 6 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Be creative when you do this! Bad guys want lots of different things. Bring out your inner bad character! Now ask yourself: "How can I accomplish my bad goal”? As thinking like an attacker needs experience, it is an opportune time to involve network security guys. A short history in the academic literature. An early paper on abuse cases at ACSAC in Abuse cases are not as commonly used as they should be. Be creative when you do this! Bad guys want lots of different things. Bring out your inner bad character! Now ask yourself: "How can I accomplish my bad goal”? As thinking like an attacker needs experience, it is an opportune time to involve network security guys. A short history in the academic literature. An early paper on abuse cases at ACSAC in Abuse cases are not as commonly used as they should be.

Software Security 7 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Security is not a feature that can be added to software. Security is an evolving property of a system, not a feature. Because security is not a feature, it can't be “attached on" after other software features are codified. –Nor can it be "patched in" after attacks have occurred in the field. Sometimes this involves making explicit tradeoffs when specifying system requirements. –A non-easy-to-use authentication procedure! Security is not a feature that can be added to software. Security is an evolving property of a system, not a feature. Because security is not a feature, it can't be “attached on" after other software features are codified. –Nor can it be "patched in" after attacks have occurred in the field. Sometimes this involves making explicit tradeoffs when specifying system requirements. –A non-easy-to-use authentication procedure! Security Is Not a Set of Features

Software Security 8 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Attackers are not standard-issue customers. Bad people who want your SW to act in some unanticipated way to their benefit. If the development process doesn't address unexpected or abnormal behavior, then attacker has plenty of raw material to work with. Attackers are creative, but we can be sure that some well-known locations will always be probed in the course of attacks: boundary conditions, edges, intersystem communication, and system assumptions. We can do this by asking and answering some critical questions: –What assumptions are implicit in our system? –What kinds of things would make our assumptions false? –What kinds of attack patterns will an attacker may use? Unfortunately, system creators rarely make the best security analysts for their own systems. Attackers are not standard-issue customers. Bad people who want your SW to act in some unanticipated way to their benefit. If the development process doesn't address unexpected or abnormal behavior, then attacker has plenty of raw material to work with. Attackers are creative, but we can be sure that some well-known locations will always be probed in the course of attacks: boundary conditions, edges, intersystem communication, and system assumptions. We can do this by asking and answering some critical questions: –What assumptions are implicit in our system? –What kinds of things would make our assumptions false? –What kinds of attack patterns will an attacker may use? Unfortunately, system creators rarely make the best security analysts for their own systems. What You Can't Do

Software Security 9 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 The simplest, most practical method for creating abuse cases is usually through a process of informed brainstorming. Formal methods are often unnecessary in the real world. A more practical approach that covers a lot of ground more quickly involves forming brainstorming teams that combine security and reliability experts with system designers. Abuse is always possible at the places where legitimate use is possible. Brainstorming involves: –a careful look at all user interfaces (including environment factors) –as well as functional security requirements and considers –what things most developers assume a person can't or won't do. These can'ts and won'ts take many forms, such as: –"Users can't enter more than 50 characters because the JavaScript code won't let them." –"The user doesn't understand the format of the cached data. They can't modify it." Attackers, unfortunately, make can'ts and won'ts happen with some regularity. The simplest, most practical method for creating abuse cases is usually through a process of informed brainstorming. Formal methods are often unnecessary in the real world. A more practical approach that covers a lot of ground more quickly involves forming brainstorming teams that combine security and reliability experts with system designers. Abuse is always possible at the places where legitimate use is possible. Brainstorming involves: –a careful look at all user interfaces (including environment factors) –as well as functional security requirements and considers –what things most developers assume a person can't or won't do. These can'ts and won'ts take many forms, such as: –"Users can't enter more than 50 characters because the JavaScript code won't let them." –"The user doesn't understand the format of the cached data. They can't modify it." Attackers, unfortunately, make can'ts and won'ts happen with some regularity. Creating Useful Abuse Cases

Software Security 10 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 System architects and project managers often respond to the very idea of abuse cases by claiming, "But no one would do these things.“ These claims are correct if the worldview is limited to legitimate users. Virtually any system that has value, can be abused. System architects and project managers often respond to the very idea of abuse cases by claiming, "But no one would do these things.“ These claims are correct if the worldview is limited to legitimate users. Virtually any system that has value, can be abused. no one would do these things …

Software Security 11 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Unfortunately, abuse cases are only rarely used in practice even though the idea seems natural. The next slide …………………. Abuse cases are to be built by a team of Requirements people and Security analysts (RAs and SAs). This team starts with –a set of requirements, –a set of standard use cases (or user stories), and –a list of attack patterns. These raw material are combined by a process to create abuse cases. Two critical activities of abuse case development are: –Creating anti-requirements –Creating an attack model Unfortunately, abuse cases are only rarely used in practice even though the idea seems natural. The next slide …………………. Abuse cases are to be built by a team of Requirements people and Security analysts (RAs and SAs). This team starts with –a set of requirements, –a set of standard use cases (or user stories), and –a list of attack patterns. These raw material are combined by a process to create abuse cases. Two critical activities of abuse case development are: –Creating anti-requirements –Creating an attack model Abuse Case Development

Software Security 12 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010

Software Security 13 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 In developing a software system, thinking explicitly about the things that you don't want your software to do is as important as the things that you do want. Very close to the requirements, called anti-requirements. Anti-requirements provide how a malicious user, attacker, competitor can abuse your system. Anti-requirements determine what happens when a functionality goes away. E.g.: “what happens in the absence of the crypto module” in a system with the requirement of encrypting some data while being written on disk. Abuse cases based on anti-requirements lead to stories about what happens in the case of failure, especially security related failure. In developing a software system, thinking explicitly about the things that you don't want your software to do is as important as the things that you do want. Very close to the requirements, called anti-requirements. Anti-requirements provide how a malicious user, attacker, competitor can abuse your system. Anti-requirements determine what happens when a functionality goes away. E.g.: “what happens in the absence of the crypto module” in a system with the requirement of encrypting some data while being written on disk. Abuse cases based on anti-requirements lead to stories about what happens in the case of failure, especially security related failure. Creating Anti- Requirements

Software Security 14 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 An attack model through explicit consideration of known attacks or attack types. Attack patterns are extremely useful. To create an attack model: –Select those attack patterns relevant to your system. Build abuse cases around those attack patterns. –Include anyone who can gain access to the system, as threats must encompass all potential sources of danger to the system. The process in the previous slide, results in a number of useful artifacts. The activities are designed to create a list of threats and their goals (a "proper threat model"), a list of relevant attack patterns, and a unified attack model. These are all side effects of the anti-requirements and attack model activities. The process creates a set of ranked abuse cases--what your system does under those attacks most likely to be experienced. This is a process that requires extensive use of your black hat. An attack model through explicit consideration of known attacks or attack types. Attack patterns are extremely useful. To create an attack model: –Select those attack patterns relevant to your system. Build abuse cases around those attack patterns. –Include anyone who can gain access to the system, as threats must encompass all potential sources of danger to the system. The process in the previous slide, results in a number of useful artifacts. The activities are designed to create a list of threats and their goals (a "proper threat model"), a list of relevant attack patterns, and a unified attack model. These are all side effects of the anti-requirements and attack model activities. The process creates a set of ranked abuse cases--what your system does under those attacks most likely to be experienced. This is a process that requires extensive use of your black hat. Creating an Attack Model

Software Security 15 Secure SW Development (SSD) Grad. Course (R. Jalili & M.S. Dousti ) – Fall 2010 Determining the can'ts and won'ts is often difficult for those who think only about positive features. Some guidance exists in the form of attack patterns. Attack patterns can be used to guide abuse case development. Determining the can'ts and won'ts is often difficult for those who think only about positive features. Some guidance exists in the form of attack patterns. Attack patterns can be used to guide abuse case development. Summary: Abuse Cases Are Useful