1. Method Using Command Abstraction Library for Iterative Testing Security of Web Applications
Suji Munetoh, The Graduate University for Advanced Studies & IBM
Nobuzaka Yoshioka, National Institute of Informatics

2. Introduction
Web applications are exposed to a variety of application-level attacks
MITRE has catalogued the vulnerabilities (CVE) and weaknesses (CWE)
Efforts to secure software have been undertaken
Security Development Lifecycle, Secure by Design, and Secure by Default
Vulnerabilities exist due to human error, configuration errors, workarounds, and quick evolution of software

3. Introduction
Security-related issues could ideally be hidden from the developers
Security assurance is difficult and is involved in non-functional requirements
Novel methods and automation is needed for awareness of recent issues in security
The waterfall-model of software development aligns with conventional security assurance
security requirements, secure design, and security testing

4. Agile Development
Agile development is common and generally done with small teams and a limited budget
Security assurance is achieved through numerous tools and approaches
A composite tool better matches the needs of Agile development
This tool ideally should be built with Agile development to respond to changes
The authors propose an automated tool to improve the security of Agile Web development

5. Application Framework
They develop a method to split the target application into the application logic and the framework
They distinguish between the two based on the lifecycle and maturity
A different approach is used for each
Application Logic – utilize static analysis (control and data flow analysis)
Framework - an abstraction library identifies flaws and security features

6. Application Framework
Splitting the program reduces the effort in the static analysis of code during Agile development
The abstraction library enables knowledge-sharing of security among the developers of the application
They test the method and tool in regard to finding vulnerability in Ruby on Rails applications and its feasibility

7. Agile Web Application Development
Leveraging open-source software, developers can create software with less effort rapidly
The Agile Manifesto (2001) has influenced various development efforts
Increased the speed at which software has been developed using an iterative cycle
Application frameworks for web can be developed using Python, Ruby, and PHP
Paper focuses on Ruby on Rails

8. Web Application Vulnerabilities
Authors identified 74 CVEs between 2006 and 2013 in web applications that use Ruby on Rails
Most of them are implementation vulnerabilities that can be found using static analysis
The design vulnerabilities generally are not detected by static analysis
Generally require security requirements

9. CVEs with CWE in Ruby on Rails Apps

10. Web Application Vulnerabilities
Known vulnerabilities get patched during upgrades
Despite this, the number of vulnerabilities (design, implementation, and misc.) has increased over the examined time period
Security testing with security requirements would reduce the number of vulnerabilities in software at the development phase

11. Web Application Vulnerabilities

12. Security Assurance
Previous research has shown that the standard approach to security assurance does not match well with Agile Development
Applying human-intensive methods (threat analysis and code review)
Documentation for security is critical to share knowledge
Open-source software tends to not have sufficient documentation and security guides

13. Security Assurance
Model-driven security expresses security and design requirements clearly
To use the approach requires time and knowledge which makes it a difficult for Agile development
Methods to identify vulnerabilities: code review, penetration test, static and dynamic analysis
Automated tests require little human involvement and have less cost

14. Security Testing
Penetration testing involves attacking the system and evaluating the results
Static analysis can achieve 100% code coverage since it works at the source code level
Precision can be an issue resulting in false-positives
Need to validate the results
Dynamic behavior of web applications complicates the data flow analysis

15. Security Testing
Control flow analysis can be useful for identifying application behavior vulnerabilities such as access control
Data flow analysis can help identify data injection attacks and lack of input sanitization
Vulnerability scanner (e.g., Nessus or Metasploit) can be used to assess security on a live system
It is difficult for Agile development where the team is small and resource-constrained

16. Security Testing
Larger teams tend to have more security tools accessible to them than smaller teams
Dynamic analysis shows the runtime behavior, but it can be difficult to get the coverage desired
Static analysis fits more with Agile development in regard to coverage and time

17. Research Questions
RQ1: How Can Security Issues Related to Web Application Development be Handled in a Unified Manner?
They suggest building a unified tool
RQ2: How Can Such a Security-Testing Tool be Developed?
Needs to stay current with the threat landscape for web apps

18. Proposed Method
Automated tool that identifies vulnerabilities (design and implementation) via static analysis
Creates a model of the code and can ascertain if it meets security properties

19. Separation of Security Test Target
Web application frameworks tend to be more compatible with code-centric and test-drive Agile development
Model generation can be automated and built from the source code
Target application is divided into two parts dependent of the lifecycle of the code
Code being actively developed
Libraries and external code
Both have security assessments that are different

20. Separation of Security Test Target
Division of code reduces the scope of static analysis and puts more focus on the application code
Examines the framework for security-related behaviors and clarifies them

21. Model Generation and Security Assessment

22. Example Controller Code

23. Command Abstraction Library (CALib)
Security functions are abstracted to commands used by the application
Security characteristics are organized by the CALib and it also generates the model
CALib parses the code and generates an Abstract Syntax Tree
The generated model is examined for vulnerabilities using the security requirements with the CALib definitions
The security requirements need to be clearly stated to be evaluated

24. Static Analysis
Static analysis evaluates the requirements and creates warning for violations of the requirements
Potential vulnerability in the application code or an error in model generation
Results need to be investigated to ensure it is not a false positive

25. CALib Types of Commands
Control Flow Commands (CFCs): Commands that generate a control flow
Data Flow Commands (DFCs): Commands that generate a data flow
Security Commands (SCs): Commands that provide a security function
Risky Commands (RCs): Commands that require security precautions

26. CALib
For the model, the CFCs display the transition between states and the DFCs display the input and output for data
RCs require special attention since they are a security sensitive sink
For an identified RC, an SC needs to used to protect it if it can contain untrusted input

27. Application Behavior Model
The primary goal of the static analysis is to ascertain whether the “sinks” are properly handled
Identify and examine
Three types of sinks
Application domain problems (cross-site scripting XSS)
Problems resulting from RCs (SQL injection)
Any problem that requires a clear definition for security requirements (access control)

28. Application Behavior Model And Commands

29. Automated Tool
Authors developed a “RailroadMap” automated tool in the Ruby scripting language for the Ruby on Rails Web Application framework (13,184 LOC)
Command line tool that has commands for steps in the assessment
CALib generates the behavior model (control and data flow)
A command is a transition for the model and uses the CFG generated from the source code

30. Automated Tool
Variables in the code that appear in the view can be sources or sinks in the data flow model
Depends on the quality of the parser and its scope
CALib generates the Model-View-Controller (MVC) model and the ASTs
State transition model contains control and data flows
It will identify the state transitions and the data input and data output

31. CALib Output Command information is obtained from CALib
Authors suggest using a HTML table for the security review
The tool may produce incomplete output (missing transitions or data flow) due to code complexity
Model can be manually altered to add these in
CALib can be extended to support new commands

32. Iterative Development Process
CALib needs to be accurate and complete since it will report an error for an undefined command
CALib is coded in Ruby and uses JSON output
Agile development can be applied to CALib
It can be tested against known vulnerable code from public sources (CVEs, GitHub)
The tool should support a partial slice of the program for modeling due to dependencies where source is not available

33. Maintenance of CALib
Commands specific to the application can be added to the library and updated to improve the completeness of the tool

34. CALib for Ruby On Rails

35. Evaluation
For evaluating R1, they tested the ability of the tool to identify vulnerabilities in the RailsGoat application
RailsGoat is a test application that contains known vulnerabilities and used as a benchmark for testing
For evaluating R2, they gauged the viability of test- driven development of the RailroadMap test tool

36. WebGoat CWE and Mitigation

37. Tool Evaluation
The tool will identify the RCs and suggest the appropriate places of SCs for mitigation
A requirement sink is split into the subtypes of domain and application
Domain sinks are reflected by the model (e.g., user input not reflected into external output (XSS)
Application sinks are specific to the application
Requires an oracle to detect this sink to determine whether it is developer-intended or not (access control)
Configuration issues can be detected by using a whitelist for proper configuration

38. Security Tool Development
They evaluated the tool’s ability to adapt to new problems in an Agile environment while both are being developed in parallel
They successfully generated test cases for the vulnerabilities from the RailsGoat application except for A9 (depends on package version)
The sizes of the test cases were of a manageable size for tool maintenance

39. Discussion Almost 100% detection of sinks in the RailsGoat application
Once a sink has been identified, they examined if the tool could identify the appropriate countermeasure in place or suggest a safe one if it is missing
If the tool does not correctly identify that a countermeasure is in place for a sink, then it is a false positive

40. Discussion
Warning message is composed of severity and confidence (high, medium, and low)
A larger amount of test cases should be used to improve the modeling and detection
Errors found by the tool are dependent on logic and coding style
Documents for security requirements and definitions for application model generation is a required manual step
Tool can be used for regression testing through iterations of development

41. Related Work
Andrews et al. proposed using a Finite State Machine (FSM) to create models from web applications’ source code and user-defined constraints
Can suffer from state explosion due to fine-grained FSMs
Sprenkle et al. used execution traces and abstract test cases to generate a navigation model for Java web applications
Few available security-focused static analysis tools exist for Ruby on Rails

42. Related Work
Chaudhuri and Foster created a symbolic execution tool to identify vulnerabilities (XSS, CSRF, session manipulation, and unauthorized access) in Ruby on Rails web apps
Near and Jackson created Derailer which uses symbolic execution by using the Ruby runtime and allows the user to creatie specifications for the security policy of the application
Various researchers have proposed adding security activities to development methodologies (Agile, XP, etc.) to raise awareness and secure the software

43. Conclusion
Created RailroadMap which improves security of Ruby on Rails web applications during the development process
Elicit security requirements
Implement security features
Testing security
Well suited to Agile development
Evaluation of the tool showed that it can detect vulnerabilities