bioRxiv Science⌕ Search

Biology subjects

Rautenstrauch, P.

Publications and source records attributed to Rautenstrauch, P..

4 recordsLinked to original sources

Metrics Matter: Why We Need to Stop Using Silhouette in Single-Cell Benchmarking

Current-day single-cell studies comprise complex data sets affected by nested batch effects caused by technical and biological factors, relying on advanced integration methods. Silhouette is an established metric for assessing clustering results, comparing within-cluster cohesion to between-cluster separation, and adaptations of it have emerged as the dominant choice to evaluate the success of these integration methods. However, silhouettes assumptions are often violated in single-cell data integration scenarios. We demonstrate that silhouette-based metrics can neither reliably assess batch effect removal nor biological signal conservation and are thus inherently unsuitable for data with (nested) batch effects. We propose alternative, robust evaluation strategies that enable accurate integration method assessment and call to update benchmarking practices.

bioinformatics↗

Liam tackles complex multimodal single-cell data integration challenges

Multi-omics characterization of single cells holds outstanding potential for profiling gene regulatory states of thousands of cells and their dynamics and relations. How to integrate multimodal data is an open problem, especially when aiming to combine data from multiple sources or conditions containing biological and technical variation. We introduce liam, a flexible model for the simultaneous horizontal and vertical integration of paired single-cell multimodal data. Liam learns a joint low-dimensional representation of two concurrently measured modalities, which proves beneficial when the information content or quality of the modalities differ. Its integration accounts for complex batch effects using a tuneable combination of conditional and adversarial training and can be optimized using replicate information while retaining selected biological variation. We demonstrate liams superior performance on multiple multimodal data sets, including Multiome and CITE-seq data. Detailed benchmarking experiments illustrate the complexities and challenges remaining for integration and the meaningful assessment of its success.

bioinformatics↗

Multimodal single cell data integration challenge: results and lessons learned

Biology has become a data-intensive science. Recent technological advances in single-cell genomics have enabled the measurement of multiple facets of cellular state, producing datasets with millions of single-cell observations. While these data hold great promise for understanding molecular mechanisms in health and disease, analysis challenges arising from sparsity, technical and biological variability, and high dimensionality of the data hinder the derivation of such mechanistic insights. To promote the innovation of algorithms for analysis of multimodal single-cell data, we organized a competition at NeurIPS 2021 applying the Common Task Framework to multimodal single-cell data integration. For this competition we generated the first multimodal benchmarking dataset for single-cell biology and defined three tasks in this domain: prediction of missing modalities, aligning modalities, and learning a joint representation across modalities. We further specified evaluation metrics and developed a cloud-based algorithm evaluation pipeline. Using this setup, 280 competitors submitted over 2600 proposed solutions within a 3 month period, showcasing substantial innovation especially in the modality alignment task. Here, we present the results, describe trends of well performing approaches, and discuss challenges associated with running the competition.

bioinformatics↗

Identifying interpretable gene-biomarker associations with functionally informed kernel-based tests in 190,000 exomes

Here we present an exome-wide rare genetic variant association study for 30 blood biomarkers in 191,971 individuals in the UK Biobank. We compare gene-based association tests for separate functional variant categories to increase interpretability and identify 193 significant gene-biomarker associations. Genes associated with biomarkers were ~ 4.5-fold enriched for conferring Mendelian disorders. In addition to performing weighted gene-based variant collapsing tests, we design and apply variant-category-specific kernel-based tests that integrate quantitative functional variant effect predictions for missense variants, splicing and the binding of RNA-binding proteins. For these tests, we present a statistically powerful and computationally efficient combination of the likelihood-ratio and score tests that found 36% more associations than the score test alone. Kernel-based tests identified 13% more associations than their gene-based collapsing counterparts and had advantages in the presence of gain of function missense variants. We introduce local collapsing by amino acid position for missense variants and use it to interpret associations and identify potential novel gain of function variants in PIEZO1. Our results show the benefits of investigating different functional mechanisms when performing rare-variant association tests, and demonstrate pervasive rare-variant contribution to biomarker variability.

genetics↗