Automated Parser Generation for High-Speed NIDS Hongyu Gao Clint Sbisa

Motivation Processing speed is crucial concern for NIDS/NIPS Limited by rate of parsing packets Inefficient parsing leads to slow speeds and bottlenecks

Current Solutions Binpac Declarative language and compiler Designed to simplify task of constructing complex protocol parsers Constructs a full parsing tree

Current Solutions Netshield Integrate high-speed protocol parser to provide fast parsing speed Parsers are manually written, which is tedious work and error-prone

Proposed Solution A protocol parser generator Read the protocol specification Output the parser for the specific protocol The parser is aware of matching The parser focuses on the fields needed by matching and skip unnecessary fields

Automated parser generation? Proposed Solutions Comparison table Automated parser generation? Yes No Fast parsing Our solution Netshield parser Binpac parser

Design Principles The parsing process should avoid recursive calls Parse trees are not used in parsing phase Skip unneeded information After parsing one field, the parser should be able to quickly jump to the next necessary field

Detailed design The parser consists of three parts A pair of buffer pointers A field table ( key data structure) A table pointer

Detailed design on field table Metadata Field type Field value Field length Garbage length Next field Field 1 Field 2 … Field n

Detailed Design on field table There are six columns in our current design Field metadata: A structure contains field name, field length and other metadata. Field type: An enum to mark if the field is directly used in matching (type 1) or used to parse other fields that need matching ( type 2) or others. Field value: Start and end pointers in the buffer. Field length: Function to obtain bytes in the field. Garbage length: Length of unnecessary fields-- used to skip fields. Next field: A method to decide the next field in the table.

How to realize the system It is difficult to determine the number of necessary fields from the very beginning. We adopt a two-phase approach. Generate the table of all fields specified. Compress the generated table to produce the table actually used in parsing.

How to realize the system The value of each column is computed in the following way: Metadata: Obtain metadata from hierarchical protocol structure in memory ( in parser generation). Obtain length value from the function to decide field length ( during parsing). Field type: Refer to both the rule set and the protocol specification to determine the field type ( in parser generation) Field value: Depend on buffer pointer and function to compute field length ( during parsing)

How to realize the system Field length: Note that this attribute is a function to compute the value of field length. It is not a simple number itself. The function can be fixed referring to protocol specification. Garbage length: In the table generation phase one, garbage length equals to 0 for every field. Next field: For fix-order fields, the next field can be determined in parser generation by search in the hierarchical protocol structure. However, for branch cases, the function to decide the next field is defined in the protocol specification.

How to realize the system Compress the phase-one table to get the final table All type 1 and type 2 fields are to be kept. They are called necessary fields Starting from each necessary fields, all fix-ordered, consecutive unnecessary fields should be merged into its garbage length, until it reaches an unnecessary field that is involved in the computation of any necessary fields.

Questions? Suggestions?