1. Python Stateful Parsing
Peter Wad Sackett

2. Stateful parsing – a text file example
Imagine a file with this content
I am a data file.
Some of my lines don’t matter to you.
Other lines contain data you want to extract, like DNA sequence.
These data span several lines.
The lines with the data can be identified by a specific pattern.
Green
ATCGTCGATGCATCGATCGATCATCGATCGTGATAGCTACGTACGT
ACTACGTCAGTCATGCTCTGTCGTACGCATGATAGCTCGTACGTCG
GTAGACCGCTACGATGCACCACACAGCGCGAATACTAGC
Red
As can be seen the sequence is between the green and the red line.
Some number data
End of file

3. Stateful parsing - method
Stateful parsing is reading a file line by line with the intention of collecting some data from the file by observing a state.
The state is cheked by a flag (variable) which is either True or False. The flag is initially False, denoting that we have not seen the green line, which tells us that the data comes now. When we see/pass the green line the flag is set to True. It is now time to collect the data. When we pass the red line the flag is set back to False indicating no more data.
Here is some pseudo code for the process:
(data, flag) = (’’, False)
for line in file:
if line is green: flag = True
if flag == True: data += line
if line is red: flag = False

4. Stateful parsing – real python code
Real code example
# statements opening a file
(data, seqflag) = (’’, False)
for line in file:
if line == ”Red\n”:
seqflag = False
if seqflag:
data += line[:-1]
if line == ”Green\n”:
seqflag = True
raise ValueError(”Format is not right. Can’t trust result”)
# Statements closing file
By changing the order of the 3 if’s, the green line and/or the red line will be considered as part of the collected data or not.