1. An assessment framework for Intrusion Prevention System (IPS)

2. Universal Assessment framework for IPS
1. Background
2. Theoretical frameworks
3. Methodological debates
4. Objectives
5. Proposed Methods
6. Contributions

3. Background of research project
Universal Assessment
Market
Requirement
Deployment
Intrusion prevention system (IPS)
Timescale
Cost
Security
Assessment
Requirement

4. Accurate assessment of IPS
Why IPS
Old security system based on Firewall + IDS can not meet enterprise level security requirement
IPS offer an active protection compared to the passive protect such as firewall, IDS
A well designed IPS product/solution can reduce the loss
IPS has less than 5 years history, it is developing
Accurate assessment of IPS
Over 15 main product lines, Over 25 IPS solutions, and the number is increasing
Complexity of the IPS
Great diversity of mechanism
Different compatibility and capability
Security market is expanding for IPS
Assessment Framework
Vender’s own assessment
Third party assessment
Universal standard assessment framework

5. Market Requirement Enterprise End user Research organisation
Assessment for their own products R & D
Marketing policy
Requirement analysis
End user
Compare the product
Choose the suitable product and solution
Research organisation
Accurate and authoritative assessment
Research

6. Testing Standard and Cost
Apart from general system measurement and standards, most solution venders, even in the case of independent testing authorities have their own testing environment and assessment methodology
The accuracy, extent and capacity of the data analysis of assessment systems and assessment frameworks needs significant improvement
Setting up a test lab
Hardware
Software
Software license
Administrative overhead

7. Essential improvement for IPS
Functionality
Stability
Reliability
Scalability
Compatibility
Usability
High Performance
Easy Deployment
Low Cost
The essential improvement on particular areas and such areas will affect both IPS systems itself and assessment measurements.
Basic standard for Universal assessment framework

8. Current Assessment Situation
Assessment methodology and organisation
Assessment carried out by products/solutions vender, sales channels, relevant industrial media
Problem:
Sales and Marketing more than evaluation
Assessment methodology based on their own standard
No third party authorise testing lab
Non-profit making organisation such as OSSTMM (Open Source Security Testing Methodology Manual), NSS group
Different assessment theory and methodology
Suitability is varies
Testing methodology is based on general security measurement, not particularly for IPS
Hard for none technical professionals to make judgment

9. Current Assessment Situation
Requirements
A common assessment framework which will be compatible with most IPS systems and give an accurate and authoritative evaluation report of their performance
A framework that will apply to the whole evaluation for different IPSs based on different infrastructures and it will offer reliable, accurate and authoritative systematic evaluation tools

10. Theoretical Frameworks
Current testing method and frameworks
manual testing toolsets
network-based or application-based automated assessment
consultant penetration testing services
Challenge:
The procedure is currently reliant upon several pieces of software and
Reliant on varying levels of IT professionals’ experience and expertise
Lacking a standardised methodology to manage the process

11. Remedy for the challenge
The remedy is the development of standardised analysis tools and methodologies.
These will lead to:
The production of key factors of assessment
Consistent data collection methods
Efficiency of compiling databases and analysing and reporting upon the data generated from IPS
Different results produced by using different testing methodologies

12. Possibility of developing framework
Feature of the proposed framework:
the most widely used
peer-reviewed
comprehensive security testing methodology
Similar methodologies and other relevant standardized testing and assessment methods:
OSSTMM
Computer Security Resource Center security testing systems
NSS Group
ISECOM
Icsalabs
This is an opportunity for developing a standardised assessment framework for IPS

13. Methodological debates
From technical point of view:
Whether it is possible to build up such an assessment framework for IPS
From practical point of view:
Is it possible and necessary to build up an assessment framework for IPS which can apply to different network infrastructures, different IPS designs and implementations, and different end users irrespective of technical or specialist knowledge

14. Possibility Measurements
Assessment Methods
Essential Knowledgebase
Universal
Assessment
Framework
Development
of IPS
Development of Assessment method
Assessment Framework
Methodology debates

15. Aim
The aim of this study is to develop an efficient, standardized, accurate and authoritative assessment framework for Intrusion Prevention System (IPS).
Researching current methods, determining commonality, differences and shortcomings of current assessment frameworks for IPS
Reviewing standard evaluation techniques in other related areas in order to formalize an alternative method to meet the criteria of assessment framework for IPS
Developing a testing framework by which the effectiveness of the above can be evaluated.

16. Objectives
To clarify current intrusion prevention system (IPS) concepts, mechanism and problems. (review stage)
Range of techniques
Measurement of key factors for IPS
Compare various solutions delivered from different companies, especially for the testing of reports from their own labs
etc.
To clarify the IPS assessment system
Software: collect and collate data, analysis, generate report and provide user interface
Hardware: in collaboration with software, supporting functionality of software
Deployment methodology: design and deployment of IPS, distributed architecture of IPS and mechanism of IPS

17. Objectives Research work of theory for common assessment framework
A method for identifying commonality, differences and variability at requirement level
A suitable approach for measuring the priority of common and variable entities
An efficient assessment framework for relative product lines
Devise new, and adapt existing, methods to understand commonality, variability and perform commonality analysis
Apply common assessment framework for IPS
To test and examine the assessment framework for IPS and get the conclusion.

18. Proposed Methods Literature review Data collection and analysis
Assessment framework design, development, testing and evaluation

19. Contributions
Building up assessment framework for IPS which can apply to different IPS products and solutions based on different network environments and requirements for the network security level. Provide an impartial alternative to the subjective assessment and unauthoritative assessment from services providers. And this research theoretically can make important contributions to the running of assessment frameworks, in practice.
Methodology of multiple criteria assessment
Based on comparisons with different assessment methodologies and the mechanism of IPSs, this research argues that assessment framework for IPSs is in fact influential and does play a significant role as provide accurate and authoritative evaluation report. It also attempts to be a starting point for further research in the field of impact assessment frameworks of current evaluation methodologies

20. Contributions Universality of assessment framework Review IPS system
Suitability for end-user
Review & compare current assessment framework
Monitor the development of IPS and IPS assessment industry
Development methodology for multi-criteria assessment framework

21. Major Milestone Literature Review Evaluation of research work
Assessment framework
Literature Review
Thesis write-up