Method Using Command Abstraction Library for Iterative Testing Security of Web Applications Suji Munetoh, The Graduate University for Advanced Studies & IBM Nobuzaka Yoshioka, National Institute of Informatics

Introduction Web applications are exposed to a variety of application-level attacks MITRE has catalogued the vulnerabilities (CVE) and weaknesses (CWE) Efforts to secure software have been undertaken Security Development Lifecycle, Secure by Design, and Secure by Default Vulnerabilities exist due to human error, configuration errors, workarounds, and quick evolution of software

Introduction Security-related issues could ideally be hidden from the developers Security assurance is difficult and is involved in non-functional requirements Novel methods and automation is needed for awareness of recent issues in security The waterfall-model of software development aligns with conventional security assurance security requirements, secure design, and security testing

Agile Development Agile development is common and generally done with small teams and a limited budget Security assurance is achieved through numerous tools and approaches A composite tool better matches the needs of Agile development This tool ideally should be built with Agile development to respond to changes The authors propose an automated tool to improve the security of Agile Web development

Application Framework They develop a method to split the target application into the application logic and the framework They distinguish between the two based on the lifecycle and maturity A different approach is used for each Application Logic – utilize static analysis (control and data flow analysis) Framework - an abstraction library identifies flaws and security features

Application Framework Splitting the program reduces the effort in the static analysis of code during Agile development The abstraction library enables knowledge-sharing of security among the developers of the application They test the method and tool in regard to finding vulnerability in Ruby on Rails applications and its feasibility

Agile Web Application Development Leveraging open-source software, developers can create software with less effort rapidly The Agile Manifesto (2001) has influenced various development efforts Increased the speed at which software has been developed using an iterative cycle Application frameworks for web can be developed using Python, Ruby, and PHP Paper focuses on Ruby on Rails

Web Application Vulnerabilities Authors identified 74 CVEs between 2006 and 2013 in web applications that use Ruby on Rails Most of them are implementation vulnerabilities that can be found using static analysis The design vulnerabilities generally are not detected by static analysis Generally require security requirements

CVEs with CWE in Ruby on Rails Apps

Web Application Vulnerabilities Known vulnerabilities get patched during upgrades Despite this, the number of vulnerabilities (design, implementation, and misc.) has increased over the examined time period Security testing with security requirements would reduce the number of vulnerabilities in software at the development phase

Web Application Vulnerabilities

Security Assurance Previous research has shown that the standard approach to security assurance does not match well with Agile Development Applying human-intensive methods (threat analysis and code review) Documentation for security is critical to share knowledge Open-source software tends to not have sufficient documentation and security guides

Security Assurance Model-driven security expresses security and design requirements clearly To use the approach requires time and knowledge which makes it a difficult for Agile development Methods to identify vulnerabilities: code review, penetration test, static and dynamic analysis Automated tests require little human involvement and have less cost

Security Testing Penetration testing involves attacking the system and evaluating the results Static analysis can achieve 100% code coverage since it works at the source code level Precision can be an issue resulting in false-positives Need to validate the results Dynamic behavior of web applications complicates the data flow analysis

Security Testing Control flow analysis can be useful for identifying application behavior vulnerabilities such as access control Data flow analysis can help identify data injection attacks and lack of input sanitization Vulnerability scanner (e.g., Nessus or Metasploit) can be used to assess security on a live system It is difficult for Agile development where the team is small and resource-constrained

Security Testing Larger teams tend to have more security tools accessible to them than smaller teams Dynamic analysis shows the runtime behavior, but it can be difficult to get the coverage desired Static analysis fits more with Agile development in regard to coverage and time

Research Questions RQ1: How Can Security Issues Related to Web Application Development be Handled in a Unified Manner? They suggest building a unified tool RQ2: How Can Such a Security-Testing Tool be Developed? Needs to stay current with the threat landscape for web apps

Proposed Method Automated tool that identifies vulnerabilities (design and implementation) via static analysis Creates a model of the code and can ascertain if it meets security properties

Separation of Security Test Target Web application frameworks tend to be more compatible with code-centric and test-drive Agile development Model generation can be automated and built from the source code Target application is divided into two parts dependent of the lifecycle of the code Code being actively developed Libraries and external code Both have security assessments that are different

Separation of Security Test Target Division of code reduces the scope of static analysis and puts more focus on the application code Examines the framework for security-related behaviors and clarifies them

Model Generation and Security Assessment

Example Controller Code

Command Abstraction Library (CALib) Security functions are abstracted to commands used by the application Security characteristics are organized by the CALib and it also generates the model CALib parses the code and generates an Abstract Syntax Tree The generated model is examined for vulnerabilities using the security requirements with the CALib definitions The security requirements need to be clearly stated to be evaluated

Static Analysis Static analysis evaluates the requirements and creates warning for violations of the requirements Potential vulnerability in the application code or an error in model generation Results need to be investigated to ensure it is not a false positive

CALib Types of Commands Control Flow Commands (CFCs): Commands that generate a control flow Data Flow Commands (DFCs): Commands that generate a data flow Security Commands (SCs): Commands that provide a security function Risky Commands (RCs): Commands that require security precautions

CALib For the model, the CFCs display the transition between states and the DFCs display the input and output for data RCs require special attention since they are a security sensitive sink For an identified RC, an SC needs to used to protect it if it can contain untrusted input

Application Behavior Model The primary goal of the static analysis is to ascertain whether the “sinks” are properly handled Identify and examine Three types of sinks Application domain problems (cross-site scripting XSS) Problems resulting from RCs (SQL injection) Any problem that requires a clear definition for security requirements (access control)

Application Behavior Model And Commands

Automated Tool Authors developed a “RailroadMap” automated tool in the Ruby scripting language for the Ruby on Rails Web Application framework (13,184 LOC) Command line tool that has commands for steps in the assessment CALib generates the behavior model (control and data flow) A command is a transition for the model and uses the CFG generated from the source code

Automated Tool Variables in the code that appear in the view can be sources or sinks in the data flow model Depends on the quality of the parser and its scope CALib generates the Model-View-Controller (MVC) model and the ASTs State transition model contains control and data flows It will identify the state transitions and the data input and data output

CALib Output Command information is obtained from CALib Authors suggest using a HTML table for the security review The tool may produce incomplete output (missing transitions or data flow) due to code complexity Model can be manually altered to add these in CALib can be extended to support new commands

Iterative Development Process CALib needs to be accurate and complete since it will report an error for an undefined command CALib is coded in Ruby and uses JSON output Agile development can be applied to CALib It can be tested against known vulnerable code from public sources (CVEs, GitHub) The tool should support a partial slice of the program for modeling due to dependencies where source is not available

Maintenance of CALib Commands specific to the application can be added to the library and updated to improve the completeness of the tool

CALib for Ruby On Rails

Evaluation For evaluating R1, they tested the ability of the tool to identify vulnerabilities in the RailsGoat application RailsGoat is a test application that contains known vulnerabilities and used as a benchmark for testing For evaluating R2, they gauged the viability of test- driven development of the RailroadMap test tool

WebGoat CWE and Mitigation

Tool Evaluation The tool will identify the RCs and suggest the appropriate places of SCs for mitigation A requirement sink is split into the subtypes of domain and application Domain sinks are reflected by the model (e.g., user input not reflected into external output (XSS) Application sinks are specific to the application Requires an oracle to detect this sink to determine whether it is developer-intended or not (access control) Configuration issues can be detected by using a whitelist for proper configuration

Security Tool Development They evaluated the tool’s ability to adapt to new problems in an Agile environment while both are being developed in parallel They successfully generated test cases for the vulnerabilities from the RailsGoat application except for A9 (depends on package version) The sizes of the test cases were of a manageable size for tool maintenance

Discussion Almost 100% detection of sinks in the RailsGoat application Once a sink has been identified, they examined if the tool could identify the appropriate countermeasure in place or suggest a safe one if it is missing If the tool does not correctly identify that a countermeasure is in place for a sink, then it is a false positive

Discussion Warning message is composed of severity and confidence (high, medium, and low) A larger amount of test cases should be used to improve the modeling and detection Errors found by the tool are dependent on logic and coding style Documents for security requirements and definitions for application model generation is a required manual step Tool can be used for regression testing through iterations of development

Related Work Andrews et al. proposed using a Finite State Machine (FSM) to create models from web applications’ source code and user-defined constraints Can suffer from state explosion due to fine-grained FSMs Sprenkle et al. used execution traces and abstract test cases to generate a navigation model for Java web applications Few available security-focused static analysis tools exist for Ruby on Rails

Related Work Chaudhuri and Foster created a symbolic execution tool to identify vulnerabilities (XSS, CSRF, session manipulation, and unauthorized access) in Ruby on Rails web apps Near and Jackson created Derailer which uses symbolic execution by using the Ruby runtime and allows the user to creatie specifications for the security policy of the application Various researchers have proposed adding security activities to development methodologies (Agile, XP, etc.) to raise awareness and secure the software

Conclusion Created RailroadMap which improves security of Ruby on Rails web applications during the development process Elicit security requirements Implement security features Testing security Well suited to Agile development Evaluation of the tool showed that it can detect vulnerabilities