Simonov, M. | bioRxiv Science

bioRxiv2025

Assessing genomic reproducibility of read alignment tools

Genomic research relies on accurate and reproducible computational analyses of DNA sequencing data to draw reliable biological conclusions. Read mapping, the process of aligning reads to a reference genome, is central to many applications, including variant detection and comparative genomics. While several tools have been developed for this task, genomic reproducibility1, defined as the consistency of results across replicates, remains underexplored. Here, we address this question by introducing a methodology based on synthetic replicates of sequencing data, generated by perturbing the original reads through shuffling, reverse complementing, or combined shuffling and reverse complementing. Our approach is able to simulate variability observed across sequencing runs due to differences in library preparation techniques. We evaluated the reproducibility of eight alignment tools (BWA-MEM22, Bowtie23, HISAT24, minimap25, NextGenMap6, SNAP7, SMALT7,8 and Subread9) under these perturbations using whole-genome sequencing (WGS) data. Synthetic replicates were aligned and compared to the original sample to quantify discrepancies. Mapping accuracy changes ranged from 0.0001% to 4.4% for primary reads, which are alignments not marked as secondary, supplementary, or duplicates, and up to 12.2% for high-quality primary reads. For primary reads, the percentage of reads commonly mapped in both the original and synthetic replicate ranged from 91.66% to 100% relative to the total number of mapped reads in the original dataset. High-quality filtering improved consistency, though some tools still failed to recover more than 70% of the original alignments. Within the set of common reads, the incidence of inconsistent mappings was as high as 13.53% for primary reads and 6.73% for high-quality primary reads. Bowtie 2 was fully reproducible under the shuffling replicate, Subread was fully reproducible under the reverse-complement replicate, whereas NextGenMap exhibited only minor inconsistencies. By contrast, SNAP and minimap2 displayed the most significant variability under reverse complementing. We further demonstrate that alignment inconsistencies propagate to the downstream task of calling structural genomic variants. Using Manta for structural-variant (SV) calling, we observed that Bowtie 2, HISAT2, and minimap2 maintained perfect SV concordance between original and replicate alignments, whereas other tools exhibited lower concordance, with Subread scoring only 87% concordance. In conclusion, our comprehensive evaluation demonstrates that synthetic perturbations reveal critical differences in how alignment tools handle technical variability and how these differences propagate to downstream variant analyses, underscoring the necessity of incorporating reproducibility benchmarks into the selection and validation of read mappers to ensure robust and reliable genomic interpretations.

bioinformatics↗

bioRxiv2020

Generating hard-to-obtain information from easy-to-obtain information: applications in drug discovery and clinical inference

In many important contexts involving measurements of biological entities, there are distinct categories of information: some information is easy-to-obtain information (EI) and can be gathered on virtually every subject of interest, while other information is hard-to-obtain information (HI) and can only be gathered on some of the biological samples. For example, in the context of drug discovery, measurements like the chemical structure of a drug are EI, while measurements of the transcriptome of a cell population perturbed with the drug is HI. In the clinical context, basic health monitoring is EI because it is already being captured as part of other processes, while cellular measurements like flow cytometry or even ultimate patient outcome are HI. We propose building a model to make probabilistic predictions of HI from EI on the samples that have both kinds of measurements, which will allow us to generalize and predict the HI on a large set of samples from just the EI. To accomplish this, we present a conditional Generative Adversarial Network (cGAN) framework we call the Feature Mapping GAN (FMGAN). By using the EI as conditions to map to the HI, we demonstrate that FMGAN can accurately predict the HI, with heterogeneity in cases of distributions of HI from EI. We show that FMGAN is flexible in that it can learn rich and complex mappings from EI to HI, and can take into account manifold structure in the EI space where available. We demonstrate this in a variety of contexts including generating RNA sequencing results on cell lines subjected to drug perturbations using drug chemical structure, and generating clinical outcomes from patient lab measurements. Most notably, we are able to generate synthetic flow cytometry data from clinical variables on a cohort of COVID-19 patients--effectively describing their immune response in great detail, and showcasing the power of generating expensive FACS data from ubiquitously available patient monitoring data. Bigger PictureMany experiments face a trade-off between gathering easy-to-collect information on many samples or hard-to-collect information on a smaller number of small due to costs in terms of both money and time. We demonstrate that a mapping between the easy-to-collect and hard-to-collect information can be trained as a conditional GAN from a subset of samples with both measured. With our conditional GAN model known as Feature-Mapping GAN (FMGAN), the results of expensive experiments can be predicted, saving on the costs of actually performing the experiment. This can have major impact in many settinsg. We study two example settings. First, in the field of pharmaceutical drug discovery early phase pharmaceutical experiments require casting a wide net to find a few potential leads to follow. In the long term, development pipelines can be re-designed to specifically utilize FMGAN in an optimal way to accelerate the process of drug discovery. FMGAN can also have a major impact in clinical setting, where routinely measured variables like blood pressure or heart rate can be used to predict important health outcomes and therefore deciding the best course of treatment.

bioinformatics↗