bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/501502

gencore: an efficient tool to generate consensus reads for error suppressing and duplicate removing of NGS data

Chen, S.·Zhou, Y.·Chen, Y.·Huang, T.·Liao, W.·Xu, Y.·Liu, Z.·Gu, J.
View original source ↗Export metadata ↓

Abstract

BackgroundRemoving duplicates might be considered as a well-resolved problem in next-generation sequencing (NGS) data processing domain. However, as NGS technology gains more recognition in clinical applications (i.e. cancer testing), researchers start to pay more attention to its sequencing errors, and prefer to remove these errors while performing deduplication operations. Recently, a new technology called unique molecular identifier (UMI) has been developed to better identify sequencing reads derived from different DNA fragments. Most existing duplicate removing tools cannot handle the UMI-integrated data. Some modern tools can work with UMIs, but are usually slow and use too much memory, making them not suitable for cloud-based deployment. Furthermore, existing tools rarely report rich statistical results, which are very important for quality control and downstream analysis. These unmet requirements drove us to develop an ultra-fast, simple, little-weighted but powerful tool for duplicate removing and sequence error suppressing, with features of handling UMIs and reporting informative results.\n\nResultsThis paper presents an efficient tool gencore for duplicate removing and sequence error suppressing of NGS data. This tool clusters the mapped sequencing reads and merges reads in each cluster to generate one single consensus read. While the consensus read is generated, the random errors introduced by library construction and sequencing can be removed. This error-suppressing feature makes gencore very suitable for the application of detecting ultra-low frequency mutations from deep sequencing data. When unique molecular identifier (UMI) technology is applied, gencore can use them to identify the reads derived from same original DNA fragment. gencore reports statistical results in both HTML and JSON formats. The HTML format report contains many interactive figures plotting statistical coverage and duplication information. The JSON format report contains all the statistical results, and is interpretable for downstream programs.\n\nConclusionsComparing to the conventional tools like Picard and SAMtools, gencore greatly reduces the output datas mapping mismatches, which are mostly caused by errors. Comparing to some new tools like UMI-Reducer and UMI-tools, gencore runs much faster, uses less memory, generates better consensus reads and provides simpler interfaces. To our best knowledge, gencore is the only duplicate removing tool that generates both informative HTML and JSON reports. This tool is available at: https://github.com/OpenGene/gencore

Source connections

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Chen, S., Zhou, Y., Chen, Y., Huang, T., Liao, W., Xu, Y., Liu, Z., Gu, J.. 2018-12-19. gencore: an efficient tool to generate consensus reads for error suppressing and duplicate removing of NGS data. https://doi.org/10.1101/501502

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Contrastive Learning Improves Generalization on M2OR Olfactory Binding Dataset

Olfaction-based biosensors stand to have profound impacts in fields such as medical diagnostics, national security, and drug discovery. A promising mode of vapor phase biodetection mimics volatile organic compound-olfactory receptor (VOC-OR) interactions with target-specific protein or peptide probes; however, lack of experimental coverage remains a core bottleneck. Current computational approaches aim to improve prediction of VOC-OR binding to reduce experimental burden, and current machine learning approaches fail to demonstrate a level of generalization that addresses current engineering challenges. Here, using the Molecular to Olfactory Dataset (M2OR) and Insect Olfactory Binding Protein Database (iOBPdb) datasets, data partitioning methods and their performance are evaluated on a neural network and contrastive learning framework on such VOC-OR binding datasets, varying: model training, architecture, and structure of the input data distribution. It was found that holding out certain chemical or proteomic motifs from the training data diminished model performance with a two layer neural network model. The model was able to generalize to state of the art performance (0.88 AUCROC) on every data split by applying a contrastive learning objective defined by shared molecular constituents and a stochastic sampler. This suggests the contrastive objective provides a strong inductive bias for neural networks to predict novel VOC-OR pairs. These findings highlight that model performance on pairwise biological interaction datasets can be inflated through identity leakage from constituent overlap, and emphasize the need for more robust evaluation protocols. Furthermore, we show contrastive regularization mitigates dataset degeneracy through enforcement of compatibility in the models latent geometry that may extend to other binder target-datasets where identity leakage confounds evaluation.

bioinformatics↗
bioRxiv2026

Plate design confounds donor variation and model evaluation in single-cell perturbation screens

Single-cell screens of immune cells from many donors are used to study why individuals respond differently and to benchmark response models. Each donor-perturbation combination typically occupies one well, controls are grouped, and donors share one layout, so technical effects can look biological. Using a 90-cytokine screen, two compound screens, four cohorts and simulations, we show that well effects explain most replicate-control variation in abundant cell types, controls carry a shared offset, and identical control wells differ by hundreds of differentially expressed genes at a standard threshold, about as many as two receptor-negative proteins do. Negative controls, design-aware estimators and perturbation-specific evaluation change the interpretation of four recent analyses. A deep-learning model that scores below a perturbation-agnostic predictor under standard evaluation outperforms a simple baseline on perturbation-specific differences in two of three cell types (significantly in one with stricter controls), but predicts less well the component shared with inert wells.

bioinformatics↗
bioRxiv2026

LinearRaccess: linear-time approximation of RNA accessibility without base-pair span limits

Motivation: RNA accessibility is the probability that every nucleotide in a contiguous interval is simultaneously unpaired; it is widely used to predict binding sites for RNA-binding proteins and other RNA-RNA interactions, often in the form of the opening energy, an equivalent measure on a logarithmic scale. Existing tools restrict the maximum base-pair span, discarding long-range base pairs that matter biologically. Others sample structures, introducing statistical variance, or compute accessibility exactly without a span limit, at cubic time cost. No existing method computes the accessibility of windows longer than one nucleotide deterministically in linear time while preserving long-range base pairs. We present LinearRaccess, which computes accessibility directly in linear time via beam-pruned inside-outside dynamic programming with a difference-array aggregation step, without restricting base-pair span. Results: For a sequence of length N and a fixed beam width b, LinearRaccess runs in O(N) time and O(Nb) memory. Against exact accessibility on 12,872 bpRNA-1m(90) sequences of 100-1,000 nt, its median opening-energy error is below 0.001 kcal/mol (0.14 kcal/mol for 700-1,000 nt), and it distinguishes windows of very small accessibility, including 35-nt windows, below the resolution of sampling-based estimates. LinearRaccess analyzes the full 29,903-nt SARS-CoV-2 genome in 419 seconds. Conclusion: LinearRaccess is the first method to compute the accessibility of windows longer than one nucleotide deterministically (without sampling) in linear time without restricting base-pair span. Availability: https://github.com/TakumiOtagaki/LinearRaccess, archived at Zenodo (https://doi.org/10.5281/zenodo.23598502).

bioinformatics↗