1. Automated Parser Generation for High-Speed NIDS
Hongyu Gao
Clint Sbisa

2. Motivation Processing speed is crucial concern for NIDS/NIPS
Limited by rate of parsing packets
Inefficient parsing leads to slow speeds and bottlenecks

3. Current Solutions Binpac Declarative language and compiler
Designed to simplify task of constructing complex protocol parsers
Constructs a full parsing tree

4. Current Solutions Netshield
Integrate high-speed protocol parser to provide fast parsing speed
Parsers are manually written, which is tedious work and error-prone

5. Proposed Solution A protocol parser generator
Read the protocol specification
Output the parser for the specific protocol
The parser is aware of matching
The parser focuses on the fields needed by matching and skip unnecessary fields

6. Automated parser generation?
Proposed Solutions
Comparison table
Automated parser generation?
Yes No
Fast parsing
Our solution
Netshield parser
Binpac parser

7. Design Principles The parsing process should avoid recursive calls
Parse trees are not used in parsing phase
Skip unneeded information
After parsing one field, the parser should be able to quickly jump to the next necessary field

8. Detailed design The parser consists of three parts
A pair of buffer pointers
A field table ( key data structure)
A table pointer

9. Detailed design on field table
Metadata
Field type
Field value
Field length
Garbage length
Next field
Field 1
Field 2 …
Field n

10. Detailed Design on field table
There are six columns in our current design
Field metadata: A structure contains field name, field length and other metadata.
Field type: An enum to mark if the field is directly used in matching (type 1) or used to parse other fields that need matching ( type 2) or others.
Field value: Start and end pointers in the buffer.
Field length: Function to obtain bytes in the field.
Garbage length: Length of unnecessary fields-- used to skip fields.
Next field: A method to decide the next field in the table.

11. How to realize the system
It is difficult to determine the number of necessary fields from the very beginning.
We adopt a two-phase approach.
Generate the table of all fields specified.
Compress the generated table to produce the table actually used in parsing.

12. How to realize the system
The value of each column is computed in the following way:
Metadata: Obtain metadata from hierarchical protocol structure in memory ( in parser generation). Obtain length value from the function to decide field length ( during parsing).
Field type: Refer to both the rule set and the protocol specification to determine the field type ( in parser generation)
Field value: Depend on buffer pointer and function to compute field length ( during parsing)

13. How to realize the system
Field length: Note that this attribute is a function to compute the value of field length. It is not a simple number itself. The function can be fixed referring to protocol specification.
Garbage length: In the table generation phase one, garbage length equals to 0 for every field.
Next field: For fix-order fields, the next field can be determined in parser generation by search in the hierarchical protocol structure. However, for branch cases, the function to decide the next field is defined in the protocol specification.

14. How to realize the system
Compress the phase-one table to get the final table
All type 1 and type 2 fields are to be kept. They are called necessary fields
Starting from each necessary fields, all fix-ordered, consecutive unnecessary fields should be merged into its garbage length, until it reaches an unnecessary field that is involved in the computation of any necessary fields.

15. Questions? Suggestions?