10/6/2014BCHB Edwards Sequence File Parsing using Biopython BCHB Lecture 11

10/6/2014BCHB Edwards2 Review Modules in the standard-python library: sys, os, os.path – access files, program environment zipfile, gzip – access compressed files directly urllib – access web-resources (URLs) as files csv – read delimited line based records from files Plus lots, lots more.

10/6/2014BCHB Edwards3 BioPython Additional modules that make many common bioinformatics tasks easier File parsing (many formats) & web-retrieval Formal biological alphabets, codon tables, etc Lots of other stuff… Have to install separately Not part of standard python, or Enthought biopython.org

10/6/2014BCHB Edwards4 Biopython: Fasta format Most common biological sequence data format Header/Description line >accession description Multi-accession sometimes represented accession1|accession2|accession3 lots of variations, no standardization No prescribed format for the description Other lines sequence, one chunk per line. Usually all lines, except the last, are the same length.

10/6/2014BCHB Edwards5 BioPython: Bio.SeqIO import Bio.SeqIO import sys # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "fasta"): # Print out the various elements of the SeqRecord print "\n------NEW SEQRECORD------\n" print "seq_record.id:\n\t", seq_record.id print "seq_record.description:\n\t",seq_record.description print "seq_record.seq:\n\t",seq_record.seq seqfile.close()

10/6/2014BCHB Edwards6 Biopython: Other formats Genbank format From NCBI, also format for RefSeq sequence UniProt/SwissProt flat-file format From UniProt for SwissProt and TrEMBL UniProt-XML format: From UniProt for SwissProt and TrEMBL Use the gzip module to handle compressed sequence databases

10/6/2014BCHB Edwards7 BioPython: Bio.SeqIO import Bio.SeqIO import sys # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "genbank"): # Print out the various elements of the SeqRecord print "\n------NEW SEQRECORD------\n" print "seq_record.id:\n\t", seq_record.id print "seq_record.description:\n\t",seq_record.description print "seq_record.seq:\n\t",seq_record.seq seqfile.close()

10/6/2014BCHB Edwards8 BioPython: Bio.SeqIO import Bio.SeqIO import sys # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "swiss"): # Print out the various elements of the SeqRecord print "\n------NEW SEQRECORD------\n" print "seq_record.id:\n\t", seq_record.id print "seq_record.description:\n\t",seq_record.description print "seq_record.seq:\n\t",seq_record.seq seqfile.close()

10/6/2014BCHB Edwards9 BioPython: Bio.SeqIO import Bio.SeqIO import sys # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "uniprot-xml"): # Print out the various elements of the SeqRecord print "\n------NEW SEQRECORD------\n" print "seq_record.id:\n\t", seq_record.id print "seq_record.description:\n\t",seq_record.description print "seq_record.seq:\n\t",seq_record.seq seqfile.close()

10/6/2014BCHB Edwards10 BioPython: Bio.SeqIO and gzip import Bio.SeqIO import sys import gzip # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = gzip.open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "fasta"): # Print out the various elements of the SeqRecord print "\n------NEW SEQRECORD------\n" print "seq_record.id:\n\t", seq_record.id print "seq_record.description:\n\t",seq_record.description print "seq_record.seq:\n\t",seq_record.seq seqfile.close()

What about the other "stuff" BioPython makes it easy to get access to non-sequence information stored in "rich" sequence databases Annotations Cross-References Sequence Features Literature 10/6/2014BCHB Edwards11

10/6/2014BCHB Edwards12 BioPython: Bio.SeqIO import Bio.SeqIO import sys import gzip # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = gzip.open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "uniprot-xml"): # What else is available in the SeqRecord? print "\n------NEW SEQRECORD------\n" print "repr(seq_record)\n\t",repr(seq_record) print "dir(seq_record)\n\t",dir(seq_record) break seqfile.close()

10/6/2014BCHB Edwards13 BioPython: Bio.SeqRecord import Bio.SeqIO import sys import gzip # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the FASTA file and iterate through its sequences seqfile = gzip.open(seqfilename) for seq_record in Bio.SeqIO.parse(seqfile, "uniprot-xml"): # Print out the various elements of the SeqRecord print "\n------NEW SEQRECORD------\n" print "seq_record.annotations\n\t",seq_record.annotations print "seq_record.features\n\t",seq_record.features print "seq_record.dbxrefs\n\t",seq_record.dbxrefs print "seq_record.format('fasta')\n",seq_record.format('fasta') break seqfile.close()

BioPython: Random access Sometimes you want to access the sequence records "randomly"… …to pick out the ones you want (by accession) Why not make a dictionary, with accessions as keys, and SeqRecord values? Use SeqIO.to_dict(…) What if you don't want to hold it all in memory Use SeqIO.index(…) 10/6/2014BCHB Edwards14

10/6/2014BCHB Edwards15 BioPython: Bio.SeqIO.to_dict(…) import Bio.SeqIO import sys # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Open the sequence database seqfile = open(seqfilename) # Use to_dict to make a dictionary of sequence records sprot_dict = Bio.SeqIO.to_dict(Bio.SeqIO.parse(seqfile, "uniprot-xml")) # Close the file seqfile.close() # Access and print a sequence record print sprot_dict['Q6GZV8']

10/6/2014BCHB Edwards16 BioPython: Bio.SeqIO.index(…) import Bio.SeqIO import sys # Check the input if len(sys.argv) >sys.stderr, "Please provide a sequence file" sys.exit(1) # Get the sequence filename seqfilename = sys.argv[1] # Use index to make an out of core dict of seq records sprot_index = Bio.SeqIO.index(seqfilename, "uniprot-xml") # Access and print a sequence record print sprot_index['Q6GZV8']

10/6/2014BCHB Edwards17 Exercises Read through and try the examples from Chapters 2-5 of BioPython's Tutorial. Download human proteins from RefSeq and compute amino-acid frequencies for the (RefSeq) human proteome. Which amino-acid occurs the most? The least? Hint: access RefSeq human proteins from ftp://ftp.ncbi.nih.gov/refseq ftp://ftp.ncbi.nih.gov/refseq Download human proteins from SwissProt and compute amino-acid frequencies for the SwissProt human proteome. Which amino-acid occurs the most? The least? Hint: access SwissProt human proteins from -> “Taxonomic divisions” How similar are the human amino-acid frequencies of in RefSeq and SwissProt?