1. Automatic XACML requests generation for policy testing
Antonia Bertolino, Said Daoudagh,
Francesca Lonetti, Eda Marchetti
Istituto di Scienza e Tecnologie
dell’Informazione “A. Faedo”
Consiglio Nazionale delle Ricerche

2. Agenda Access control policies and XACML language
Why a testing methodology?
An empirical evaluation
Conclusions and future works
X-CREATE Demo

3. Access Control Policies
Data and resources must be protected against unauthorized, malicious or improper usage or modification
Policies
specification
Data
Resources
- -

4. Testing the Policy Implementation
Policies
SUT
verdict
request
PDP
request
request
request
Test Suite
reply
Oracle
PDP (Policy Decision Point): evaluates the requests against the access control policies

5. XACML Policy Structure
XACML (eXtensible Access Control Markup Language)
The XACML Policy elements:
PolicySet
Policy
Target
Subjects
Resources
Actions
Environments
Rules
Esample of Policy
<Policy>
<Target>
<Subject>Mario Rossi</Subject>
<Resource>personal id</Resource>
<Action>read</Action>
</Target>
</Policy>
Example of request
<Request>
</Request>

6. X-CREATE Testing Framework
XaCml REquests derivAtion for TEsting
Implements several testing strategy:
Preliminary XPT (XML Partition Testing)
Incremental XPT
Simple Combinatorial
Hierarchical Simple
Hierarchical Incremental
Instantiated
Request
Policies
specification
Request
structure

7. Preliminary XPT Main Idea
Deriving once and for all a universally valid generic test suite of conforming requests by applying:
A variant of the Category Partition methodology
The Boundary Conditions methodology
Request
structure
XACML
Context
Schema
Conforming
test suite

8. E.G. Fixing ∞ to 3 X X
{1,2,3}
{1,2,3}

9. An Example
Example of request structure
<Request>
<Subject> </Subject>
<Resource> </Resource>
<Action> </Action>
</Request>
Issue: The maximum number of structurally different intermediate requests is of
310 * 21 =

10. 118098!!!! Too Much!!! Testing objectives:
New methodology for request structures generation (Incremental XPT)
New stopping criterion for test requests execution
New specific test strategy satisfying the stopping criterion (Simple Combinatorial)

11. Incremental XPT 36 = 729 request stuctures:
one value for the <AttributeValue>
zero to minOccurs and maxOccurs of the ResourceContent element and those of the contained <Any> element because not used in test values generation

12. Filling request structures with values
Take values from the policy under test for elements and attributes.
SubjectSet
ResourceSet
ActionSet
EnvironmentSet
An entity is a combination of 4 values taken from these sets (n-wise approach is used)

13. Toy Example <Policy> <Target>
<Subject>Mario Rossi</Subject>
<Resource>personal id</Resource>
<Action>read</Action>
</Target>
</Policy>
AttributeId
Data Type
Attribute Value
SubjectSet
Subjectid
string
Mario Rossi
ResourceSet
Resourceid
personal id
ActionSet
Actionid
read
EnvironmentSet

14. Complete Table
For robustness and negative testing random values for elements and attributes are added
AttributeId
Data Type
Attribute Value
SubjectSet
Subjectid
string
Mario Rossi S1 s2
ResourceSet
Resourceid
personal id R1 r2
ActionSet
Actionid
read A1 a2
EnvironmentSet E1 e2

15. How many entities?
Avoiding duplication derive all combinations of subject entities, resource entities, action entities and environment entities by applying:
the pair-wise combination (PW)
the three-wise combination (TW)
apply the four-wise combination (FW)
Note: The number of combinations is limited and strictly depends on the policy considered

16. Examples <Subject>s2</Subject> Example of request
<Subject>Mario Rossi</Subject>
<Subject>s2</Subject>
<Resource>p2</Resource>
<Action>read</Action>
<Enviroment>e2</Enviroment>
</Request>
Example of request
<Request>
<Subject>Mario Rossi</Subject>
<Resource>personal id</Resource>
<Action>read</Action>
</Request>
Example of request
<Request>
<Subject>s2</Subject>
<Resource>personal id</Resource>
<Action>a2</Action>
</Request>

17. Simple Combinatorial
Idea: derive as many requests as the possible combinations of the values of the subjects, resources, actions and environment of the XACML policy.
The number of combinations could be also be used as a stopping criterion for the test case generation in XPT

18. Incremental XPT vs. Simple Combinatorial
Research questions: 1st Match TSEff: adopting the proposed stopping criterion, is the fault detection of the Simple Combinatorial strategy similar to that of the Incremental XPT one? 2nd MatchTSDecr: is it possible to reduce the test suites maintaining the same level of fault detection? 3rd MatchTSIncr: is it possible to increase the Incremental XPT fault detection?

19. Rules of comparison Evaluation of the test strategies effectiveness:
Define a set of XACML policies
Apply mutation to each policy to introduce faults
Execute each set of test cases on the policy and its mutants
Establish the winner according in each match

20. XPT v.s. Simple Combinatorial
Simple cobinatorial
Incremental XPT
XPT v.s. Simple Combinatorial
1st Match TSEff:
The same number of requests for each policy
the effectiveness of the Incremental XPT is generally higher than that of the Simple Combinatorial strategy
In two cases the fault detection of the Simple Combinatorial is higher than that of Incremental XPT

21. Deep Analysis
Incremental XPT is the winner when the access decision of the policy rules depends concurrently on the values of more than one subject or resource or action or environment entity
Simple Combinatorial is the winner when the policies are very simple and the satisfiability of the policy rules depends on the combinations of a single subject, resource, action and environment entity

22. 2nd Match TSDecr: from the first request ahead till we reached the maximum reachable percentage of fault detection
Simple Combinatorial is the winner
Note: For XPT usually the maximum reachable percentage is reached with almost half of the available requests =>the stopping criterion is a good upper bound

23. XPT v.s. Simple Combinatorial
3rd Match TSIncr: the loss in the fault detection effectiveness due to the stopping criterion
Execute the full pull of available requests
percentage of mutants killed could be increased a lot
Calculate the minimum # requests for the maximum fault detection effectiveness
in most of the cases the loss of fault detection effectiveness is
around 15%.

24. Preliminary Conclusions and Future Works
A good fault detection percentage of the Incremental XPT testing strategy due to the variability of the structures of the generated requests
It is possible to reduce the number of requests
The high variability of the Incremental XPT strategy can limit its performance when policies are very simple
Homework:
Generalize the results
Consider further mutation operators
Conceive new test strategy generating requests containing all the possible combinations of more than one subject, resource, action and environment entity.

25. Thank you!