A Deep Learning Approach to Bacterial Colony Segmentation Paolo Andreini, Simone Bonechi, Monica Bianchini, Alessandro Mecocci, Franco Scarselli DIISM, Department of Information Engineering and Mathematics University of Siena bonechi@diism.unisi.it 1/18

Outline Petri Plate Analysis Synthetic Data Generation Bacterial Segmentation Results Conclusions and Future Perspectives 2/18

Petri Plate Analysis – Standard Protocol Cylindrical glass or plastic dish used to culture cells Filled with a growing media designed to allow the bacterial growth Food and beverage safety, environmental control, clinical analyses, etc. Sample Seeding Incubation Visual analysis 3/18

Petri Plate Analysis - Why bother? Hundreds of plates each day in big labs, e.g. 200-300 in Careggi Hospital, Florence, urine samples only Highly repetitive and prone to error Costly in term of both time and resources 4/18

Automatic Petri Plate Analysis Automatic Petri Plate Analysis Pipeline [1]: Image Acquisition Bacterial Segmentation Colony Count Bacterial Classification [1] P. Andreini, S. Bonechi, M. Bianchini, A. Garzelli, A. Mecocci. Automatic Image Classification for the Urinoculture Screening. Computers in Biology and Medicine, vol. 70, pp. 12–22, 2016 5/18

Bacterial Segmentation CNN 6/18

CNNs Need Data A unique existing public dataset labeled for segmentation - Haemolysis Dataset [2]: Only one type of culturing substrate Only few hundreds annotated data But… [2] M. Savardi, A. Ferrari, A. Signoroni. Automatic hemolysis identification on aligned dual–lighting images of cultured blood agar plates. Computer Methods and Programs in Biomedicine, vol. 156, pp. 13–24, 2018 7/18

Synthetic Data Generation 8/18

Streaking Simulation 9/18

Colony Modeling 𝑚 𝑥,𝑦 = 𝑝 𝑥,𝑦 − argmin 𝑟 𝑘 ( 𝑝 𝑥,𝑦 − 𝑟 𝑘 ) 10/18

Opacity Simulation 11/18

Rendering and Blending 12/18

Advantages Suitable for different seeding path Just need different probability matrix Suitable for different growing media and colony types Just need new colony patches Free Annotations Generator 13/18

Examples of Synthetic Plates 14/18

Bacterial Segmentation PSP-Network [3] based on ResNet with 50 layers Syntetic dataset: 120000 synthetic images (119000 training and 1000 validation) MicrobIA Dataset: 324 images (221 training and 103 test images) Experimental setups: Training on synthetic data only Training on real data from the MicrobIA dataset only Training on synthetic data and fine–tuning on real data from the MicrobIA dataset [3] H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia. Pyramid Scene Parsing Network. Proc. of CVPR ’17 pp. 6230–6239, 2017. 15/18

Results Experimental Setup Mean IoU Pixel Accuracy Synthetic Images 82,79 % 98,29 % Real Images 85,30 % 99,19 % Synthetic and Real Images 86,33 % 99,26 % 16/18

Real and Synthetic Images Results Original Image Train on Synthetic Images Train on Real Images Train on both Real and Synthetic Images Target 17/18

Conclusions and Future Perspectives The proposed synthetic image generator allows to: Create images that resemble the originals Create large datasets for bacterial segmentation Train a deep neural network even if a public dataset of real images is not available Future Perspectives Better estimation of the seeding distribution Use of Generative Adversarial Networks to improve the quality of the simulated images 18/18