Tox_(R)CNN: Deep Learning-Based Nuclei Profiling tool For Drug Toxicity Screening

Abstract

Toxicity is an important factor in failed drug development, and its efficient identification and prediction is a major challenge in drug discovery. We have explored the potential of microscopy images of fluorescently labeled nuclei for the prediction of toxicity based on nucleus pattern recognition. Deep learning algorithms obtain abstract representations of images through an automated process, allowing them to efficiently classify complex patterns, and have become the state-of-the art in machine learning for computer vision. Here, deep convolutional neural networks (CNN) were trained to predict toxicity from images of DAPI-stained cells pre-treated with a set of drugs with differing toxicity mechanisms. Different cropping strategies were used for training CNN models, the nuclei-cropping-based Tox-CNN model outperformed other models classifying cells according to health status. Tox-CNN allowed automated extraction of feature maps that clustered compounds according to mechanism of action. Moreover, fully automated region-based CNNs (RCNN) were implemented to detect and classify nuclei, providing per-cell toxicity prediction from raw screening images. We validated both Tox-(R)CNN models for detection of pre-lethal toxicity from nuclei images, which proved to be more sensitive and have broader specificity than established toxicity readouts. These models predicted toxicity of drugs with mechanisms of action other than those they had been trained for and were successfully transferred to other cell assays. The Tox-(R)CNN models thus provide robust, sensitive, and cost-effective tools for in vitro screening of drug-induced toxicity. These models can be adopted for compound prioritization in drug screening campaigns, and could thereby increase the efficiency of drug discovery.\n\nAuthor summaryVisualization of nuclei using different microscopic approaches has for decades allowed the identification of cells undergoing cell death, based on changes in morphology, nuclear density, etc. However, this human-based visual analysis has not been traslated into quantitative tools able to objectively measure cytotoxicity in drug-exposed cells. We asked ourselves if it would be possible to train machines to detect cytotoxicity from microscopy images of fluorescently stained nuclei, without using specific toxicity labeling. Deep learning is the most powerful supervised machine learning methodology available, with exceptional abilities to solve computer vision tasks, and was thus selected for the development of a toxicity quantification tool. Two convolutional neural networks (CNN) were developed to classify cells based on health status: Tox-CNN, relying on prior cell segmentation and cropping of nuclei images, and Tox-RCNN which carries out fully-automated cell detection and classification. Both Tox-(R)CNN classification outputs provided sensitive screening readouts that detected pre-lethal toxicity and were validated for a broad array of toxicity pathways and cell assays. Tox-(R)CNN approaches excel in affordability and applicability to other in vitro toxicity readouts and constitute a robust screening tool for drug discovery.