Quality Control & Preprocessing of Metagenomic Data EdwardsLab @ SDSU Robert Schmieder – rschmieder@gmail.com

Need for automated approach Metagenomic datasets contain 100,000s (454) or 1,000,000s (Illumina) of sequences IlluminaHiSeq 2000: currently 300 GB of data soon 2,000 GB (≈33 human genomes with 20x coverage in single sequencing run) Can not just read sequence by sequence to get an idea of your data

Basic data analysis Perform similarity search New dataset Assemble

Bad data analysis

Bad data analysis

Bad data analysis

Bad data analysis

Bad data analysis

Bad data analysis

Good data analysis New dataset

Good data analysis New dataset Quality control & Preprocessing

Good data analysis New dataset Quality control & Preprocessing Similarity search Assembly

Good data analysis New dataset Quality control & Preprocessing Similarity search Assembly

3 Tools for metagenomic data http://prinseq.sourceforge.net http://tagcleaner.sourceforge.net http://deconseq.sourceforge.net

Quality control and data preprocessing

Number and Length of Sequences

Number/Length of sequences Bad Reads should be approx. same length (same number of cycles)  Short reads are likely lower quality Good

Quality of Sequences

Linearly degrading quality across the read Trim low quality ends

Quality filtering Any region with homopolymer will tend to have a lower quality score Huseet al. found that sequences with an average score below 25 had more errors than those with higher averages Huse et al.: Accuracy and quality of massively parallel DNA pyrosequencing. Genome Biology (2007)

Low quality sequence issue Most assemblers or aligners do not take into account quality scores Errors in reads complicate assembly, might cause misassembly, or make assembly impossible

What if quality scores are not available ? Alternative: Infer quality from the percent of Ns found in the sequence Removes regions with a high number of Ns Huseet al. found that presence of any ambiguous base calls was a sign for overall poor sequence quality Huse et al.: Accuracy and quality of massively parallel DNA pyrosequencing. Genome Biology (2007)

Ambiguous bases If you can afford the loss, filter out all reads containing Ns Assemblers (e.g. Velvet) and aligners (SHAHA2, BWA, …) use 2-bit encoding system for nucleotides some replace Ns with random base, some with fixed base (e.g. SHAHA2 & Velvet = A) 2-bit example: 00 – A, 01 – C, 10 – G, 11 - T

Sequence duplicates

Real or artificial duplicate ? Metagenomics = random sampling of genomic material Why do reads start at the same position? Why do these reads have the same errors? No specific pattern or location on sequencing plate 11-35% Gomez-Alvarez et al.: Systematic artifacts in metagenomes from complex microbial communities. ISME (2009) 25

One micro-reactor – Many beads Martine Yerle (Laboratory of Cellular Genetics, INRA, France)

Impacts of duplicates False variant (SNP) calling Require more computing resources Find similar database sequences for same query sequence Assembly process takes longer Increase in memory requirements Abundance or expression measures can be wrong

Impacts of duplicates False variant (SNP) calling Require more computing resources Find similar database sequences for same query sequence Assembly process takes longer Increase in memory requirements Abundance or expression measures can be wrong

Depends on the experiment In contrast, for Illumina reads with high coverage: eliminating singletons is an easy way of dramatically reducing the number of error- prone reads

Tag Sequences

No tag MID tag WTA tags

Detect and remove tag sequences

Fragment-to-fragment concatenations

Concatenated fragments in assembled contigs

Data upload Tag sequence definition

Tag sequence prediction

Parameter definition Download results

Sequence Contamination

Principal component analysis (PCA) of dinucleotide relative abundance Microbial metagenomes Viral metagenomes

Identification and removal of sequence contamination

Contaminant identification Current methods have critical limitations Dinucleotide relative abundance uses information content in sequences  can not identify single contaminant sequences Sequence similarity seems to be only reliable option to identify single contaminant sequences BLAST against human reference genome is slow and lacks corresponding regions (gaps, variants, …) Novel sequences in every new human genome sequenced* * Li et al.: Building the sequence map of the human pan-genome. Nature Biotechnology (2010)

DeconSeq web interface Two types of reference databases Remove Retain

DeconSeq web interface (cont.)

Human DNA contamination identified in 145 out of 202 metagenomes

Conclusions Quality control and data preprocessing are very important to increase quality of downstream analysis Preprocessing depends on the experiment