1 Scalable and Effective Test Generation for Access Control Systems Ammar Masood School of Electrical & Computer Engineering Purdue University 11 th September, 2006

2 Outline Introduction Problems and Contributions – Part A Details of Proposed Solutions – Part B Conclusion and Future Work

3 Motivation and Challenges Protection of information from unauthorized access or modification and protection against denial of service to authorized users is an important security requirement Access control is one of the key security service providing the support for secure information access Desired access control objectives only achieved if the underlying implementation conforms to the policy, hence testing becomes essential Key challenge: how to devise scalable and effective test generation techniques ?

4 Requirement for Testing A number of vulnerabilities are related to design and/or coding flaws in access control modules of an application* OSVDB reports 53 vulnerabilities related to access control NVD which records CVE and CERT advisories reports 859 vulnerabilities with impact “provides unauthorized access” and type “access validation error”, 1440 for any impact Security Focus reports 80 vulnerabilities for the key word “access control” Formal verification and static or dynamic program-analysis techniques only guarantee correctness of design Testing is required to detect any faults in the implementation due to, for example, coding errors and incorrect configuration *Data as of 8/30/06

5 Conformance and Functional Testing

6 Testing Context

7 Role Based Access Control (RBAC) and Temporal RBAC RBAC is a promising approach for addressing diverse security needs of business organizations Access control in organizations is based on “roles that individual users take on as part of the organization” A role is “is a collection of permissions” Constraints are applied to all the links TRBAC extends RBAC by imposing duration constraints on user-role assignments/activations and permission-role assignments

8 Outline Introduction Problems and Contributions – Part A Details of Proposed Solutions – Part B Conclusion and Future Work

9 Contributions 1.RBAC fault model 2.Test generation for RBAC Systems 3.A Probabilistic model for fault coverage 4.An empirical evaluation 5.Test generation for TRBAC Systems Behavior modeling of TRBAC systems TRBAC conformance testing

10 1. RBAC Fault Model Required to study fault coverage of any test generation technique Proposed fault model comprises Mutation-based (simple) faults Non-mutation (malicious) faults Behavioral conformance used to study the fault model

11 2. Test Generation for RBAC Systems Requirements :- Effectiveness – fault detection effectiveness measured with respect to RBAC fault model Scalability – the cost of test generation and execution Existing research – Chandarmouli and Blackburn functional testing technique for Discretionary Access Control Effectiveness not considered Not amenable for fault coverage analysis

12 Proposed Solution Set of conformance testing procedures with varying cost and effectiveness Procedure A : Complete-FSM based Procedure B : Heuristics based Procedure C : Constrained Random Test Selection (CRTS) strategy based Procedure A is most effective – complete fault coverage for simple faults and a class of malicious faults – and most costly Cost and effectiveness of Procedures B and C varies with the heuristic considered for test generation and the length of test cases in the CRTS suite

13 Proposed Solution (continued) Functional Testing Required to ensure that ACUT conforms to all RBAC policies Proposed methodology is based on policy meta test set White box coverage criteria used as a feed back mechanism to establish correctness of ACUT functionality The functional testing technique is generic in that it can be used for TRBAC systems

14 3. A Probabilistic Model for Fault Coverage Requirement A mechanism for analytically comparing fault coverage of heuristics and CRTS strategy based test generation techniques Existing research Petrenko et. al. use mutation based approach to access fault coverage of tests for FSM’s One-to-one relation between faults and structural mutants Not suitable for our analysis because of many-to-many relation between RBAC/TRBAC faults and structural mutants

15 Proposed Approach Coverage matrix used to model relation between FSM and RBAC faults Faults exhibited randomly across the FSM transitions Fault coverage analytically studied for two general cases of fault distribution (uniform and non-uniform) Simulation:- To study fault coverage of test generation techniques for fault distributions achieved as mix of uniform and non-uniform distributions High coverage of all techniques for uniform case Coverage drops as distribution limits to complete non-uniform case Coverage directly proportional to the number of transitions in the test suite

16 4. Empirical Evaluation To study cost and effectiveness of use of all the procedures in functional testing of an RBAC system Based on X-GTRBAC prototype system X-GTRBAC consists of Policy initializer Policy enforcer (ACUT) Fault detection effectiveness measured through program mutation and manual injection of malicious faults Program mutants manually associated with RBAC faults (simple faults) Cost measured in terms of total number of state queries performed in the execution of a test suite

17 Results Procedure A most effective and most costly Heuristics and CRTS strategy perform equally well for simple faults but heuristics lag CRTS strategy in detecting malicious faults Effectiveness of CRTS increases as length of tests included in the suite increases, cost also increases but is significantly less then that of Procedure A Reasons: Heuristics by design fail to consider a holistic view of the system Simple faults are exhibited across much higher number of transitions as compared to malicious faults, thus easier to detect CRTS randomly select paths of fixed length from complete FSM, thus as length of tests increases there are more chances of inclusion of higher length paths in the CRTS test suite

18 Recommendations to Practitioner Although only Procedure A provides complete fault coverage it could be prohibitively expensive CRTS strategy provides the balance between cost and effectiveness Reaffirmation of usefulness of white-box criteria to enhance tests generated using black-box approach Malicious faults likely to be missed easily by the heuristics As exhaustive testing not a viable option, functional testing requires white-box criteria as a feed back mechanism to determine the stopping point

19 Comparison with Simulation Results Fault coverage results for the case of uniform fault distribution in the simulation are close to case study results for simple faults Given a test generation technique, the analytic result of fault coverage for uniform fault distribution may be used as a predictor of its effectiveness in detecting simple faults Wide disparity between coverage results for the simulation and for the case study for malicious faults Logical result as malicious faults are injected with malicious intent, thus can not be modeled with uniform distribution

20 5. Test Generation for TRBAC Systems Require effective and scalable test generation technique How to measure effectiveness? TRBAC fault model (extensions in RBAC fault model) Scalability ? Determined by the size of the test suite (size of model) Why can’t existing approaches for test generation be directly used for TRBAC test generation? Techniques for RBAC system not usable as simple FSM’s cannot capture real-time considerations Solution – use Timed Input Output Automata (TIOA) to model TRBAC TIOA based test generation techniques Symbolic clustering of states – scalable but effectiveness not measurable State characterization set based (Timed-Wp method) – effective but not at all scalable TIOA transformation to FSM (se-FSA based) – effective and scalable

21 Proposed Approach

22 Behavior modeling of TRBAC systems Requirement Model correctly specify the behavior implied by the TRBAC specification TRBAC model (TRBAC M ) is based on TIOA Two options in constructing TRBAC M Construct a single monolithic model Divide the system into parts – compositional construction TRBAC M = UR M || PR M TRBAC M is proved to correctly model the TRBAC specification

23 TRBAC conformance testing Key steps Transformation of TRBAC M into se-FSA Constructing the test tree corresponding to the se-TRBAC M Use of an Integer Programming (IP) based approach to generate the conformance test suite Fault detection effectiveness Provides complete fault coverage by virtue of correctness of TRBAC M and the correlation between TRBAC, TIOA and se-FSA faults Heuristics can be used to reduce the model size and thus the size of the corresponding test suite May result into reduced fault detection effectiveness, can be analytically studied for cases of fault distribution using the probabilistic model

24 Outline Introduction Problems and Contributions – Part A Details of Proposed Solutions – Part B Conclusion and Future Work

25 Conformance Relation Based on behavioral conformance Specified using the two conditions, which informally imply that ACUT assigns (deassigns) and activates (deactivates) a role only if such assignment (deassignment) and activation (deactivation) is allowable by the current policy in effect assigns (deassigns) a set of permissions to (from) a role only if allowable by the current policy in effect ignores ill-formed requests

26 RBAC Fault Model Conformance between ACUT and ACUT implies absence of any faults in the ACUT i.e. faults in P Conformance testing of ACUT can thus be considered as verifying that P does not belong to set of faulty policies RBAC fault model defines the set of faulty policies Obtained using mutation based approach [Petrenko et.al.] Three types of operators used for mutating the elements of RBAC P Set mutation operators Element modification operators Rule mutation operators

27 Malicious Faults Counter based A specific count of events leads to fault I/O based Faults based on malformed requests Sequence-based A specific sequence of events leads to fault

28 Conformance Testing Procedures Behavior implied by a policy expressed as an FSM. Heuristics applied to scale down the model. Use the W-method, or its variant, to generate tests from the complete (Procedure A) or scaled down model (Procedure B) or randomly select paths of fixed length from the complete model (Procedure C)

29 Sample FSM Two users, one role. Only one user can activate the role. Number of states≤3 2. AS AS 21 AC 11 AC 21 AS 21 AS 11 AC 21 AC 11 AS 11 DS 11 DS 21 DC 11 DS 21 DC 11 DS 11 DS 21 DS 11 DC 21 DS 21 DS 11 DS 21 AS: assign. DS: De-assign. AC: activate. DC: deactivate. X ij : do X for user i role j.

30 Heuristics H1: Separate assignment and activation H2: Use FSM for activation and single test sequence for assignment H3: Use single test sequence for assignment and activation H4: Use a separate FSM for each user H5: Use a separate FSM for each role H6: Create user groups for FSM modeling.

31 Reduced Models AS DS 21 DS AS 21 DS 11 DS 21 AC AC 21 DC 21 DC 11 AC 21 AC 11 Assignment MachineActivation Machine Heuristic 1 AS DS 11 AC 11 DC 11 AC 11 AS DS 21 AC 21 DC 21 AC 21 Heuristic 4 User u 1 MachineUser u 2 Machine

32 Procedure C: CRTS Strategy Constructs a pool RTi of n random tests. Lengths of all tests in the pool RTi is same, i.e. i which is selected to be comparable with the length of longest test generated using Procedure A The total number of tests n is selected based on comparison with the maximum number of tests generated using the heuristics (Procedure B) Construct five test suites RTi1,…., RTi5 by randomly selecting fixed number p of tests from RTi p empirically chosen based on economical or statistical criterion

33 Probabilistic Model for Fault Coverage State observability assumed Based on Coverage matrix C x, x  {H0, H1,…, RTi} Visibility of faults among transitions is given by  x =b. C x where b is a identity row vector of length j Fault Coverage (FC x ) is computed as where

34 Boundary Cases of Fault Distribution 1.|F|=j=|T H0 |, such that one-to-one correspondence between faults and transitions, FC x = # of transitions in x/j If x1 covers more transitions then x2  FC x1 > FC x2 2.Single fault f with equal probability of being exhibited across any transition t   T H0 Fault coverage of x is now the probability of detecting f using x

35 General Cases of Fault Distribution Case A: The total number of transition across which each fault f is exhibited is uniformly distributed Case B: Total number of faults is more than 1, each fault f has equal probability of being exhibited across any transition t   T H0

36 Simulation Five cases of fault distribution Cases 0 and 4 – same as Cases A and B Cases 1, 2 and 3 – respectively correspond to cases in which 75%, 50% and 25% of faults are uniformly exhibited (as per Case 0) rest as per Case 4 Metrics used for comparison of testing generation techniques Average fraction of faults detected Probability of detecting all faults p(F) Setup 10,000 iterations 5 values of fault density 0.01, 0.05, 0.1, 0.2 and 0.5

37 Results : Average fraction of Faults Detected Common trend for all cases of fault distribution Expected as faults are independently and identically exhibited High coverage for all techniques for Case 0 As fault distribution limits to Case 4, coverage reduces dramatically for techniques with less number of transitions in their test suites

38 Results : Probability of Detection of all Faults p(F) reduces considerably with increase in fault density Expected as p(F) is the product of probabilities for detection of individual faults As fault distribution limits to Case 4, the exponential term in p(F) corresponding to Case 4 dominates No test generation technique other than the complete FSM based, provides guarantee of detecting all faults Solution – use white box adequacy criteria for test enhancement

39

40 Empirical Evaluation : Setup Study carried out using the proposed functional testing methodology Stopping criterion – complete coverage of simple faults Policy meta set – comprises two policies Meta test sets – corresponding to the three procedures Test generation techniques used H3, H4 and H5 heuristics RT4, RT6, RT10 and RT tests in each test suite RTij

41 Empirical Evaluation : Results

42 Empirical Evaluation and Simulation Results Comparison

43 TRBAC Fault Model Conformance relation similar to the one for RBAC systems Addition of a condition to consider temporal conformance RBAC fault model extended by changing the application of rule mutation operator, result is addition of three temporal faults

44 Timed Input Output Automata (TIOA)

45 TRBAC Modeling TRBAC M = UR M || PR M UR M =UR b1 || ur UR b2 || ur, …,|| ur UR bk, three types of UR b ’s corresponding to user-role (UR) pairs with 1.Explicit assignment information 2.No explicit assignment and implicit activation 3.No explicit assignment but implicit activation L0 L1 ?AS(u 1,r 1,t 1 ) x 1 :=0 x 1 =t 1 !DS(u 1,r 1 ) L0 UR assign (u 1,r 1 )=0, UR active (u 1,r 1 )=0 L1 UR assign (u 1,r 1 )=1, UR active (u 1,r 1 )=0 L2 UR assign (u 1,r 1 )=1, UR assign (u 1,r 1 )=1 ?AC(u 1,r 1,t 2 ) L2 ?AC(u 1,r 1,t 2 ) x 2 :=0 x 2 =t 2 !DC(u 1,r 1 ) x 1 =t 1 !DS(u 1,r 1 )

46 TRBAC Modeling (continued) PR M =PR b1 || pr PR b2 || pr, …,|| pr PR bk, two types of PR b ’s corresponding to permission-role (PR) pairs with 1.Explicit assignment information 2.No explicit and implicit assignment Example: Three permissions p 1, p 2 and p 3, three roles r 1, r 2 and r 3, r 2  I r 3 p 2 r 1, p 3 r 1 and p 1 r 2 explicit assignment

47 Sample TRBAC M Example policy with a user u 1 two roles {r 1, r 2 } Constraint: u 1 cannot be simultaneously assigned to both roles No permissions considered thus TRBAC M = UR b (u 1,r 1 ) || ur UR b (u 1,r 2 )

48 se-FSA Transformation [Khoumsi] Three types of events Input events – input actions and/or clock resets Output events – output actions and/or clock expirations Complex events – mix of above two L0 L1 ?AS(u 1,r 1,t 1 ) x 1 :=0 x 1 =t 1 !DS(u 1,r 1 ) ?AC(u 1,r 1,t 2 ) L2 ?AC(u 1,r 1,t 2 ) x 2 :=0 x 2 =t 2 !DC(u 1,r 1 ) x 1 =t 1 !DS(u 1,r 1 ) t 1 =4 and t 2 =2 4<x 1 x 1 <x 2 - l0l0 q2q2 0<x 1 0<x 2 - l0l0 0<x 1 <4 2<x 2 2<x 1 -x 2 <4 l1l1 q0q0 q3q3 0<x 1 <4 x 1 <x 2 - l1l1 0<x 1 <4 0<x 2 <2 0<x 1 -x 2 <4 l2l2 4<x 1 0<x 2 <2 2<x 2 -x 1 <4 l0l0 4<x 1 2<x 2 - l0l0 q1q1 q5q5 q4q4 q6q6 ?AS(u 1,r 1 ), Set(x 1,4) Exp(x 1,4), !DS(u 1,r 1 ) ?AC(u 1,r 1 ), Set(x 2,2) Exp(x 1,4), !DS(u 1,r 1 ) Exp(x 1,4), !DS(u 1,r 1 ) Exp(x 2,2), !DC(u 1,r 1 ) Exp(x 2,2) ?AS(u 1,r 1 ), Set(x 1,4) Exp(x 1,4), Exp (x 2,2) !DS(u 1,r 1 ) Exp(x 2,2),?AS(u 1,r 1 ), Set(x 1,4)  se-FSA

49 Test Generation From se-TRBAC M se-TRBAC M deterministic and finite state W-method can thus be used for test generation Assumed location observability – tests constructed from test tree (Tr) Tr constructed so that all terminals correspond to accepting states of se-TRBAC M Tr represents paths in se-TRBAC M, Given a path p t in Tr, A test sequence is constructed by associating all edges e  p t with monotonically increasing time stamps Temporal constraints determined by the Set and Exp events along edges of p t

50 How to Construct a Test Sequence? Corresponding to path p t1 The temporal constraints can be represented as p t1 Formulate as an IP to control the minimum resolution dt i For k=0.1 the solution would be Conformance Test Suite (CTS) constructed by finding feasible time stamps for all test sequences

51 How to Apply a Test Sequence ? Used the architecture proposed by Khoumsi Given a test sequence, following semantics considered for time stamps associated with: Inputs – time at which Test-Controller should generate the corresponding input for the ACUT and the Clock-Handler Outputs – ACUT will pass the given sequence if outputs by the ACUT and Clock-Handler and the states match input output Set(c,k) Exp(c,k) Test System State query State info Test-ControllerClock-Handler ACUT

52 Fault Coverage of CTS Determined using the relation between TRBAC, TIOA and se-FSA fault models (F M ) TRBAC F M TIOA F M se-FSA F M correlated with Output, Transfer, missing and extra location faults in TIOA F M have similar representation in se-FSA Time constraint restriction/widening faults – output/transfer faults Clock reset faults not directly comparable – shown to be detectable by CTS Implies CTS detects all TRBAC Faults

53 Conclusion Proposed a unified framework for scalable and effective conformance and functional testing of RBAC and TRBAC systems Effectiveness studied using the proposed RBAC and TRBAC fault models Scalability achieved using proposed conformance testing procedures with varying cost Proposed a probabilistic model for fault coverage to analytically evaluate fault detection effectiveness of proposed conformance test generation techniques for various cases of fault distribution Performed an empirical study to evaluate the cost and effectiveness of proposed procedures in functional testing of a prototype RBAC system

54 Future Work Test generation for TRBAC systems Extending the temporal constraints in TRBAC specification Extension of TRBC fault model Conducting an empirical evaluation Validation of global meta-policy in collaborative environments Regression testing techniques for access control systems

55 Backup Slides

56 Advantages of RBAC Allows efficient security management through role hierarchy and administrative roles Principle of least privilege allows minimizing damage due to misuse of privilege Separation of duty constraints prevent fraud Role specific SoD constraint disallows conflicting roles to be accessed by same user User specific SoD constraint disallows conflicting user to access same role Encompasses traditional discretionary and mandatory policies

57 Functional Testing Methodology

58 Many-to-Many Relation between RBAC and FSM faults AS 11 A transfer fault UR1 and UR2 faults AS AS 21 AS 11 DS 21 f 1 : UR1 fault transfer fault

59 Behavioral Conformance

60 RBAC Fault Model – Simple Faults Relation between FSM and RBAC Fault Model

61 Fault Coverage of H4 for Boundary Case 1 AS AS 21 AC 11 AC 21 AS 21 AS 11 AC 21 AC 11 AS 11 DS 11 DS 21 DC 11 DS 21 DC 11 DS 11 DS 21 DS 11 DC 21 DS 21 DS 11 DS 21 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t7t7 t8t8 t9t9 t 10 AS DS 11 DC 11 AC 11 AS DS 21 DC 21 AC 21 t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t7t7 t8t8 t9t9 t 10 FSM(P) H4: M u1 and M u2