Flow Cytometry and Reproducible Analysis Cliburn Chan Department of Biostatistics and Bioinformatics, DUMC

Reproducible Analysis Can someone in a different lab replicate your results? Can someone else in your lab replicate your results? Can you replicate your own results – 6 months later? – When FlowJo goes from version 10.0 to 11.0? – When your lab catches fire and all your computers melt into toxic waste?

Complexity of flow analysis Experimental design Running the experiment Raw data (FCS files) Compensation Transformation Gating strategy Gates  MFI and relative frequencies Statistical analysis – e.g. outcome correlation

Experimental design Is randomization done correctly? Is the sample size sufficient? Is there an SOP for annotating the experiment? – MIATA – MiFlowcyt What is the informatics strategy to ensure that data is recorded accurately and backed-up safely?

Running the experiment Stuff I know little about … Janet and Jennifer will teach in this workshop – Instrument calibration – Bridging studies – Reagent qualification – Use of appropriate biological controls – Use of appropriate technical controls

Raw data (FCS files) Is there a file naming SOP that is followed? Is there an SOP for recording FCS metadata? – Channel labels – fluorochrome, antibody, FMO

Inconsistent annotation example

Compensation, transformation and gating strategy Compensation is Real = Spillover -1 × Observed Transformation is complicated – can think of as linear (low values) and log (high values) Gating strategy is hard to replicate, but can be stored as a template and “re-used” with tweaking Compensation, transformation and gating should be done on a per-batch and not per-file basis Would recommend storing workspace containing this data in both.jo and.xml formats

Working with statisticians At some point, a statistician is likely to be asked to analyze your data. This can lead to much unhappiness. Statisticians do not like Excel – The first thing they will try to do is export to a CSV or delimited file, for import into SAS or R – If this is difficult to do, they will not like you

Excel rules for happy statisticians 1 worksheet = 1 table 1 cell = 1 value Data/metadata = comprehensive & consistent Formatting = None Validation = Yes

1 worksheet = 1 table A table has column headers and a number of rows and nothing else – it is RECTANGULAR Do not put more than 1 table in a worksheet Do not use non-rectangular tables Example of good worksheet

1 worksheet = 1 table

1 cell = 1 value Easy to filter by tube, sample or subject Easy to write validation rules or lookup table

1 cell = 1 value ID column has 3 different values Need to do text parsing to recover information – very error prone

Data: column names Consistent column names across worksheets – Singlets/Lymphocytes – Singlet/Lymphs – Singlets / Lymphocytes – Singlets/Lymphoctyes Use full gating path for column name – Singlets/Lymphocytes/Viable/CD4+/CM/IFN+

Data: What to record Better to have more data than less data – Sample type (PBMC, whole blood) – Recovery – Viability Better to have basic than derived data – Counts better than relative frequencies Keep link to raw data for reproducibility – Path to FCS and workspace files on server Use special indicator for missing data (e.g. NAN), not zero Use as many columns as you need and name them sensibly and consistently

Data: Versioning Do not change the data in the worksheet once it has been handed to statistician. If there are errors that must be corrected, make a new copy, label the filename with date and version, and send that to statistician – ArcticRatExperiment_07May2013_Version01.xlsx – ArcticRatExperiment_17May2013_Version02.xlsx

Formatting Don’t do it. Avoid putting information via: – Highlighting – Fancy spacing – Different fonts and font effects – Merging cells – Comments Will it survive a round-trip from Excel to CSV and back again?

Formatting - Before

Formatting - After Comments are lost Highlighting is lost Bad cell formatting is lost Merged cells become missing information

Summary of Reproducible Analysis Know what you are doing from PBMC to Excel SOPs are important Annotation is important Excel is OK if you use NONE of its features Keep all necessary data in the same place Keep a remote backup Talk with your statistician

Biologist talks to Statistician