bioRxiv Science⌕ Search

Biology subjects

Chikhi, R.

Publications and source records attributed to Chikhi, R..

At least 19 recordsLinked to original sources

Sketching the SRA: experiment-wide similarity search across 4.8 million metagenomes

Recent efforts have demonstrated that systematic analysis of the millions of metagenomes in the Sequence Read Archive (SRA) enables groundbreaking biological discoveries. However, SRA-scale comparison of whole metagenomes has remained elusive. Here, we introduce the most comprehensive resource to date for experiment-to-experiment analysis of all SRA metagenomes, comprising FracMinHash sketches (470GB), hypergen sketches (19GB), and an all-pairs Jaccard similarity matrix (90GB), which we release online at https://github.com/RolandFaure/SRA-metagenome-sketches-databases. These resources enable a wide range of analyses, which we demonstrate with several applications. First, we use the FracMinHash sketches to uncover temporal trends, finding that the amount of newly discovered microbial sequence continues to grow, rather than saturate or shrink. Second, we use the hypergen sketches, which enable nearest-neighbor search, to identify co-assembly candidates and thereby improve assembly quality. Third, we use the similarity matrix to analyze all SRA metagenomes, identifying duplicated submissions and over 50K candidates for metadata correction. Together, our resources unlock systematic analysis of metagenomic data at unprecedented scale.

bioinformatics↗

The Genome In A Bottle HG002 assembly-based variant benchmark set enables comprehensive benchmarking of small and structural variants.

High-accuracy, high-resolution benchmark data are fundamental to improving the accuracy and generalizability of sequencing technologies and associated algorithms. Most existing benchmarks for DNA sequencing rely on mapping to a single linear reference, omitting inaccessible and highly repetitive complex genomic regions. Here, we present the Genome in a Bottle (GIAB) Consortium's HG002 v5.0q benchmark set, a more comprehensive variant benchmark derived from a fully phased, highly curated, and accurate diploid assembly of HG002. Our approach leverages recent advances in long-read sequencing and assembly to provide confident small- and structural-variant calls across previously inaccessible regions, including segmental duplications, homopolymers, tandem repeats, and complex immune loci. The v5.0 benchmark significantly expands the callable genome relative to v4.2.1, adding x200 Mbp of sequence and increasing variant detection by 18 % for small variants and 3-fold for structural variants compared to previous benchmarks. The benchmark regions include more variants in repetitive regions, including 1.2 x segmental duplications, 1.8 x tandem repeats, and 2.0 x homopolymers, compared to v4.2.1. Separate and combined small- and structural-variant benchmarks are available for three versions of the human reference genome (GRCh37, GRCh38, and CHM13). We validate the reliability of this benchmark set by comparing multiple sequencing technologies and variant calling pipelines and curating differences. The v5.0q benchmark provides a critical resource for the development and validation of next-generation sequencing technologies and analysis methods.

genomics↗

Lossless compression of k-mer matrices enabling random row access

Genomic search engines such as Logan-Search index petabytes of sequencing data as large binary matrices, called k-mer matrices, where each row encodes the presence of a k-mer across thousands to millions of genomic samples. Logan-Search contains a petabyte of binary matrices, and storing them is expensive, yet compression must not prevent fast random access to any matrix row at query time. We present kmcomp, a lossless compression method for k-mer matrices that satisfies these competing requirements. Block compression partitions the matrix into fixed-size row blocks, each compressed independently; block start positions are stored in an Elias-Fano encoded array, enabling O(1) random access to any block. To improve compressibility without introducing additional decompression steps, we introduce the{pi} -compression: a column reordering that groups similar samples together by solving the Traveling Salesman Problem via a nearest-neighbor heuristic. We accelerate this heuristic with a novel variant of the vantage-point tree, the masked vp-tree, which dynamically prunes nearest-neighbor search space. On three (meta)genomic datasets, kmcomp achieves compression ratios of 1.3 to 5.4;{pi} -compression further improves these to 1.5 to 51.3. Applied to the Logan-Search petabyte-scale index, compression reduces storage by approximately half, and{pi} -compression adds a further 13% gain. Query overhead remains modest: queries of hundreds of nucleotides incur an absolute latency increase of{approx} 100 ms, and highly compressed indexes can match uncompressed query times thanks to reduced disk reads.

bioinformatics↗

Petabase-scale Papillomavirus Discovery

Freely available nucleic acid sequencing databases have accumulated to a vast archive of genetic diversity, in excess of 50 petabase-pairs from tens of millions of experiments. Together, these data constitute a digital survey of Earths genome. However, the richness of biological information contained within these repositories remains largely unexplored, in large part owing to the technical challenges of analyzing petabytes of data. Recently, Logan completed the sequence-assembly and compression of 27 million sequencing libraries from the Sequence Read Archive (SRA), and here, we systematically search Logan-SRA to reveal the global diversity of the DNA-based Papillomaviruses (PVs). In a single [~]10-hour alignment-based search against the Logan assemblage, we independently re-identified 65% of the 992 PVs recorded within the NCBI Virus database, a body of work representing over five decades of PV characterization. We further expand the diversity of PVs by 34%, identifying 383 novel PV types spanning 105 associated host species, including taxa with no previously associated PVs, such as rhinoceros, voles, and grey foxes. Through integration of virus phylogeny, sample geography, and ecological metadata, we show that novel PV discovery is not directly proportional to sampling effort, and there are significant hotspots of PV biodiversity in East Africa and South America, and that undersampled biomes can yield disproportionately more novel PVs. Public sequencing repositories contain vast, unrealized biological information that is now accessible through advanced computational infrastructure. Here we lay the foundations for the efficient analysis of petabase-scale datasets for DNA virus discovery, with applications to uncovering all genetic diversity.

bioinformatics↗

Rapid modular evolution of antiviral repertoires in P2 phages and their P4 satellites: shuttling, swapping and mixing

Interactions between bacteria, bacteriophages, and their satellites are shaped by a myriad of defence and counter-defence mechanisms. Here, we identified and characterized the defence hotspots of thousands of P2-like phages and P4-like satellites to elucidate the origins and evolutionary dynamics of defence systems. Both P4 and P2 encode a broad diversity of recognizable defence systems. Defences are a substantial, yet likely underestimated, share of the elements pangenomes, as shown by novel antiviral functions discovered in P4 loci lacking known defence genes. Defence loci are very rapidly swapped, without pseudogenization, suggesting defences are replaced before becoming non-adaptive. This intense local recombination melds components of distinct systems into novel functional chimeras. Systems swap so rapidly that many elements with identical core genes have completely different defences. Surprisingly, despite P4 and P2s concomitant replication and packaging, they almost never exchange defence genes. In contrast, near identical defence systems can be found in distinct types of MGEs and in cryptic chromosomal locations. Our findings highlight P4 and P2 as mobile platforms driving the modular diversification of bacterial antiviral repertoires. Hence, bacterial defences change quickly by phage and satellite turnover, and by the quick swap of defences within these elements.

microbiology↗

Taxonomic-free metagenome GWAS to identify gut microbiome functions influencing host phenotypes

Genome-wide association studies (GWAS) have been pivotal for uncovering the genetics of human phenotypes, and there is now growing interest in applying GWAS-like methods to explore the role of the microbiome in human health. Here, we developed a taxonomy free GWAS approach that uses k-mers, i.e. DNA words of length k that can capture single nucleotide polymorphisms, insertion or deletion events, and gene presence-absence, to interrogate the gut microbiome. We applied this method to 26 traits spanning demographics, physiological measurements, health, and lifestyle in 938 healthy participants from the Milieu Interieur cohort. We generated a k-mer abundance matrix encompassing 97 million distinct k-mers. GWAS of the 26 traits identified significant associations for seven of them: age, sex, depression, appetite, cooked meat consumption, soda intake, and smoking. By modeling the correlation structure among k-mers, we identified a modest number of independent signals and conducted a comprehensive in silico functional annotation of these signals, revealing potential mechanisms of host-microbiota interaction. Overall, our analyses demonstrate that k-mers can capture biologically relevant functions shared across multiple taxa and provide a refined modeling framework that complements the standard taxonomic-based screening approach.

microbiology↗

Inverted colored de Bruijn Graph for practical kmer sets storage

Petabases of sequencing data in the Sequence Read Archive (SRA) present a significant challenge for holistic reanalysis due to their sheer volume. Recent efforts have assembled this data into terabytes of unitigs, an efficient k-mer set representation that can reduce data size by an order of magnitude. However, these unitigs were compressed on a per-accession basis, leaving substantial cross-sample redundancy unexploited. While co-compression of related samples offers high space-saving potential, existing tools lack targeted decompression: the ability to retrieve specific documents at a cost proportional to their individual sizes rather than that of the entire collection. This paper introduces the "inverted de Bruijn graph" property, formalizing the concept of efficient targeted decompression, and presents kloe, its first implementation. kloe is a compression method for large, highly similar k-mer multi-sets, such as collections of unitigs from related samples. Unlike existing approaches that map k-mers to colors (samples), kloe takes a complementary route by performing color-to-k-mer mapping, associating samples with their respective k-mer sets. This enables targeted decompression of any chosen samples k-mer content. At its core, kloe utilizes a new sequence construct called "monochromatigs," drawing on concepts from simplitigs and monotigs to achieve both significant space savings and efficient retrieval. Finally, a central aim of this work is to highlight this novel problem area, which we argue is critically understudied compared to colored de Bruijn graphs. The associated tool is available as an open source project at github.com/TimRouze/KLOE

bioinformatics↗

Anyone can be the best: Impact of diverse methodologies on the evaluation of structural variant callers

Structural variants (SVs) are medium and large-scale genomic alterations that shape phenotypic diversity and disease risk. Numerous methods have been proposed for discovering SVs, however their benchmarking has been inconsistent across studies, often resulting in contradictory findings. One of the main sources of conflicting evaluation re-sults is the lack of consistency in the SV callsets used as ground truth, ranging from curated callsets released by consortia to more recent approaches that construct callsets from high-quality telomere-to-telomere de novo haplotype assemblies. The discrepancies between benchmarks are further compounded by the choice of the reference genome (GRCh37, GRCh38, and T2T-CHM13), where using T2T-CHM13reveals a different deletion/insertion profile, indicating reduced reference bias. We evaluated the performance of several state-of-the-art SV discovery methods from long-read whole-genome sequencing data and observed substantial variation in their performance and rankings, depending on the choice of ground truth, reference genome, and genomic regions used for evaluation. Counter-intuitively, the more complete reference genome T2T-CHM13does not inherently solve the problem of SV benchmarking; instead it reveals the limitations of each detection method in complex genomic regions. The substantial variation in detection accuracy across different genomic regions calls for additional caution in downstream analyses and in drawing conclusions based on predicted SVs. These findings underscore the complexity of evaluating SV detection methods and highlight the need for careful consideration and, ideally, field-standard best practices when reporting performance metrics.

bioinformatics↗

High-quality metagenome assembly from nanopore reads with nanoMDBG

Third-generation long-read sequencing technologies, have been shown to significantly enhance the quality of metagenome assemblies. The results obtained using the highly accurate reads generated by PacBio HiFi have been particularly notable yielding hundreds of circularized, complete genomes as metagenome-assembled genomes (MAGs) without manual intervention. Oxford Nanopore Technologies (ONT) has recently improved the accuracy of its sequencing reads, achieving a per-base error rate of approximately 1-2%. Given the high-throughput, convenience and low-cost of ONT sequencing this could accelerate the uptake of long read metagenomics. However, current metagenome assemblers are optimized for PacBio HiFi data and underperform on the latest ONT data and do not scale to the large data sets that it enables. We present nanoMDBG, an evolution of the metaMDBG HiFi assembler, designed to support newer ONT sequencing data through a novel pre-processing step that performs fast and accurate error correction in minimizer-space. Across a range of ONT datasets, including a large 400 Gbp soil sample sequenced specifically for this study, nanoMDBG reconstructs up to twice as many high-quality MAGs as the next best ONT assembler, metaFlye, while requiring a third of the CPU time and memory. As a result of these advances, we show that the latest ONT technology can now produce results comparable to those obtained using PacBio HiFi sequencing at the same sequencing depth.

bioinformatics↗

Pangenome graph augmentation from unassembled long reads

Pangenomes are becoming increasingly popular data structures for genomics analyses due to their ability to compactly represent the genetic diversity within populations. Constructing a pangenome graph, however, is still a time-consuming and expensive process. A promising approach for pangenome construction consists of progressively augmenting a pangenome graph with additional high-quality assemblies. Currently, there is no method for augmenting a pangenome graph with unassembled reads from newly sequenced samples without first aligning the reads to a reference genome and performing variant calling and genotyping on the new individuals. In this work, we present the first assembly-free and mapping-free approach for augmenting an existing pangenome graph using unassembled long reads from an individual not already present in the pangenome. Our approach consists of finding sample specific sequences in reads using efficient indexes, clustering reads corresponding to the same novel variant(s), and then building a consensus sequence to be added to the pangenome graph for each variant separately. Using simulated reads based on Human Pangenome Reference Consortium (HPRC) assemblies, we demonstrate the effectiveness of the proposed approach for progressively augmenting the pangenome with long reads, without the need for de novo assembly or predicting genetic variants of the new sample. The software is freely available at https://github.com/ldenti/palss.

bioinformatics↗

Logan: Planetary-Scale Genome Assembly Surveys Life's Diversity

The breadth of lifes diversity is unfathomable, but public nucleic acid sequencing data offers a window into the dispersion and evolution of genetic diversity across Earth. However the rapid growth and accumulation of sequence data have outpaced efficient analysis capabilities. The largest collection of freely available sequencing data is the Sequence Read Archive (SRA), comprising 27.3 million datasets or 5 x 1016 basepairs. To realize the potential of the SRA, we constructed Logan, a massive sequence assembly transforming short reads into long contigs and compressing the data over 100-fold, enabling highly efficient petabase-scale analysis. We created Logan-Search, a k-mer index of Logan for free planetary-scale sequence search, returning matches in minutes. We used Logan contigs to identify >200 million plastic-degrading enzyme homologs, and validate novel enzymes with catalytic activities exceeding current reference standards. Further, we vastly expand the known diversity of proteins (30-fold over UniRef50), plasmids (22-fold over PLSDB), P4 satellites (4.5-fold), and the recently described Obelisk RNA elements (3.7-fold). Logan also enables ecological and biomedical data mining, such as global tracking of antimicrobial resistance genes and the characterization of viral reactivation across millions of human BioSamples. By transforming the SRA, Logan democratizes access to the worlds public genetic data and opens frontiers in biotechnology, molecular ecology, and global health.

bioinformatics↗

Statistical signature of subtle behavioural changes inlarge-scale behavioural assays

The central nervous system can generate various behaviours, including motor responses, which we can observe through video recordings. Recent advancements in genetics, automated behavioural acquisition at scale, and machine learning enable us to link behaviours to their underlying neural mechanisms causally. Moreover, in some animals, such as the Drosophila larva, this mapping is possible at unprecedented scales of millions of animals and single neurons, allowing us to identify the neural circuits generating particular behaviours. These high-throughput screening efforts are invaluable, linking the activation or suppression of specific neurons to behavioural patterns in millions of animals. This provides a rich dataset to explore how diverse nervous system responses can be to the same stimuli. However, challenges remain in identifying subtle behaviours from these large datasets, including immediate and delayed responses to neural activation or suppression, and understanding these behaviours on a large scale. We introduce several statistically robust methods for analyzing behavioural data in response to these challenges: 1) A generative physical model that regularizes the inference of larval shapes across the entire dataset. 2) An unsupervised kernel-based method for statistical testing in learned behavioural spaces aimed at detecting subtle deviations in behaviour. 3) A generative model for larval behavioural sequences, providing a benchmark for identifying complex behavioural changes. 4) A comprehensive analysis technique using suffix trees to categorize genetic lines into clusters based on common action sequences. We showcase these methodologies through a behavioural screen focused on responses to an air puff, analyzing data from 280,716 larvae across 568 genetic lines. Author SummaryThere is a significant gap in understanding between the architecture of neural circuits and the mechanisms of action selection and behaviour generation.Drosophila larvae have emerged as an ideal platform for simultaneously probing behaviour and the underlying neuronal computation [1]. Modern genetic tools allow efficient activation or silencing of individual and small groups of neurons. Combining these techniques with standardized stimuli over thousands of individuals makes it possible to relate neurons to behaviour causally. However, extracting these relationships from massive and noisy recordings requires the development of new statistically robust approaches. We introduce a suite of statistical methods that utilize individual behavioural data and the overarching structure of the behavioural screen to deduce subtle behavioural changes from raw data. Given our studys extensive number of larvae, addressing and preempting potential challenges in body shape recognition is critical for enhancing behaviour detection. To this end, we have adopted a physics-informed inference model. Our first group of techniques enables robust statistical analysis within a learned continuous behaviour latent space, facilitating the detection of subtle behavioural shifts relative to reference genetic lines. A second array of methods probes for subtle variations in action sequences by comparing them to a bespoke generative model. Together, these strategies have enabled us to construct representations of behavioural patterns specific to a lineage and identify a roster of "hit" neurons with the potential to influence behaviour subtly.

neuroscience↗

Exploring a large cancer cell line RNA-sequencing dataset with k-mers

Analyzing the immense diversity of RNA isoforms in large RNA-seq repositories requires laborious data processing using specialized tools. Indexing techniques based on k-mers have previously been effective at searching for RNA sequences across thousands of RNA-seq libraries but falling short of enabling direct RNA quantification. We show here that RNAs queried in the form of k-mer sets can be quantified in seconds, with a precision akin to that of conventional RNA quantification methods. We showcase several applications by exploring an index of the Cancer Cell Line Encyclopedia (CCLE) collection consisting of 1019 RNA-seq samples. Non-reference RNA sequences such as RNAs harboring driver mutations and fusions, splicing isoforms or RNAs derived from repetitive elements, can be retrieved with high accuracy. Moreover, we show that k-mer indexing offers a powerful means to reveal variant RNAs induced by specific gene alterations, for instance in splicing factors. A web server allows public queries in CCLE and other indexes: https://transipedia.fr. Code is provided to allow users to set up their own server from any RNA-seq dataset.

bioinformatics↗

Reference-free Structural Variant Detection inMicrobiomes via Long-read Coassembly Graphs

Bacterial genome dynamics are vital for understanding the mechanisms underlying microbial adaptation, growth, and their broader impact on host phenotype. Structural variants (SVs), genomic alterations of 10 base pairs or more, play a pivotal role in driving evolutionary processes and maintaining genomic heterogeneity within bacterial populations. While SV detection in isolate genomes is relatively straightforward, metagenomes present broader challenges due to absence of clear reference genomes and presence of mixed strains. In response, our proposed method rhea, forgoes reference genomes and metagenome-assembled genomes (MAGs) by encompassing a single metagenome coassembly graph constructed from all samples in a series. The log fold change in graph coverage between subsequent samples is then calculated to call SVs that are thriving or declining throughout the series. We show rhea to outperform existing methods for SV and horizontal gene transfer (HGT) detection in two simulated mock metagenomes, which is particularly noticeable as the simulated reads diverge from reference genomes and an increase in strain diversity is incorporated. We additionally demonstrate use cases for rhea on series metagenomic data of environmental and fermented food microbiomes to detect specific sequence alterations between subsequent time and temperature samples, suggesting host advantage. Our innovative approach leverages raw read patterns rather than references or MAGs to include all sequencing reads in analysis, and thus provide versatility in studying SVs across diverse and poorly characterized microbial communities for more comprehensive insights into microbial genome dynamics.

bioinformatics↗

Petascale Homology Search for Structure Prediction

The recent CASP15 competition highlighted the critical role of multiple sequence alignments (MSAs) in protein structure prediction, as demonstrated by the success of the top AlphaFold2-based prediction methods. To push the boundaries of MSA utilization, we conducted a petabase-scale search of the Sequence Read Archive (SRA), resulting in gigabytes of aligned homologs for CASP15 targets. These were merged with default MSAs produced by ColabFold-search and provided to ColabFold-predict. By using SRA data, we achieved highly accurate predictions (GDT_TS > 70) for 66% of the non-easy targets, whereas using ColabFold-search default MSAs scored highly in only 52%. Next, we tested the effect of deep homology search and ColabFolds advanced features, such as more recycles, on prediction accuracy. While SRA homologs were most significant for improving ColabFolds CASP15 ranking from 11th to 3rd place, other strategies contributed too. We analyze these in the context of existing strategies to improve prediction.

bioinformatics↗

Efficient High-Quality Metagenome Assembly from Long Accurate Reads using Minimizer-space de Bruijn Graphs

We introduce a novel metagenomics assembler for high-accuracy long reads. Our approach, implemented as metaMDBG, combines highly efficient de Bruijn graph assembly in minimizer space, with both a multi-k' approach for dealing with variations in genome coverage depth and an abundance-based filtering strategy for simplifying strain complexity. The resulting algorithm is more efficient than the state-of-the-art but with better assembly results. metaMDBG was 1.5 to 12 times faster than competing assemblers and requires between one-tenth and one-thirtieth of the memory across a range of data sets. We obtained up to twice as many high-quality circularised prokaryotic metagenome assembled genomes (MAGs) on the most complex communities, and a better recovery of viruses and plasmids. metaMDBG performs particularly well for abundant organisms whilst being robust to the presence of strain diversity. The result is that for the first time it is possible to efficiently reconstruct the majority of complex communities by abundance as nearcomplete MAGs.

bioinformatics↗

kmindex and ORA: indexing and real-time user-friendly queries in terabytes-sized complex genomic datasets

Public sequencing databases contain vast amounts of biological information, yet they are largely underutilized as one cannot efficiently search them for any sequence(s) of interest. We present kmindex, an innovative approach that can index thousands of highly complex metagenomes and perform sequence searches in a fraction of a second. The index construction is an order of magnitude faster than previous methods, while search times are two orders of magnitude faster. With negligible false positive rates below 0.01%, kmindex outperforms the precision of existing approaches by four orders of magnitude. We demonstrate the scalability of kmindex by successfully indexing 1,393 complex marine seawater metagenome samples from the Tara Oceans project. Additionally, we introduce the publicly accessible web server "Ocean Read Atlas" (ORA) at https://ocean-read-atlas.mio.osupytheas.fr/, which enables real-time queries on the Tara Oceans dataset. The open-source kmindex software is available at https://github.com/tlemane/kmindex.

bioinformatics↗

Construction and representation of human pangenome graphs

As a single reference genome cannot possibly represent all the variation present across human individuals, pangenome graphs have been introduced to incorporate population diversity within a wide range of genomic analyses. Several data structures have been proposed for representing collections of genomes as pangenomes, in particular graphs. In this work we collect all publicly available high-quality human haplotypes and constructed the largest human pangenome graphs to date, incorporating 52 individuals in addition to two synthetic references (CHM13 and GRCh38). We build variation graphs and de Bruijn graphs of this collection using five of the state-of-the-art tools: Bifrost, mdbg, Minigraph, Minigraph-Cactus and pggb. We examine differences in the way each of these tools represents variations between input sequences, both in terms of overall graph structure and representation of specific genetic loci. This work sheds light on key differences between pangenome graph representations, informing end-users on how to select the most appropriate graph type for their application.

bioinformatics↗