bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/327833

An analysis and comparison of the statistical sensitivity of semantic similarity metrics

Manda, P.·Vision, T.
View original source ↗Export metadata ↓

Abstract

1Semantic similarity has been used for comparing genes, proteins, phenotypes, diseases, etc. for various biological applications. The rise of ontology-based data representation in biology has also led to the development of several semantic similarity metrics that use different statistics to estimate similarity.\n\nAlthough semantic similarity has become a crucial computational tool in several applications, there has not been a formal evaluation of the statistical sensitivity of these metrics and their ability to recognize similarity between distantly related biological objects.\n\nHere, we present a statistical sensitivity comparison of five semantic similarity metrics (Jaccard, Resnik, Lin, Jiang& Conrath, and Hybrid Relative Specificity Similarity) representing three different kinds of metrics (Edge based, Node based, and Hybrid) and explore key parameter choices that can impact sensitivity. Furthermore, we compare four methods of aggregating individual annotation similarities to estimate similarity between two biological objects - All Pairs, Best Pairs, Best Pairs Symmetric, and Groupwise.\n\nTo evaluate sensitivity in a controlled fashion, we explore two different models for simulating data with varying levels of similarity and compare to the noise distribution using resampling. Source data are derived from the Phenoscape Knowledgebase of evolutionary phenotypes.\n\nOur results indicate that the choice of similarity metric along with different parameter choices can substantially affect sensitivity. Among the five metrics evaluated, we find that Resnik similarity shows the greatest sensitivity to weak semantic similarity. Among the ways to combine pairwise statistics, the Groupwise approach provides the greatest discrimination among values above the sensitivity threshold, while the Best Pairs statistic can be parametrically tuned to provide the highest sensitivity.\n\nOur findings serve as a guideline for an appropriate choice and parameterization of semantic similarity metrics, and point to the need for improved reporting of the statistical significance of semantic similarity matches in cases where weak similarity is of interest

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Manda, P., Vision, T.. 2018-05-22. An analysis and comparison of the statistical sensitivity of semantic similarity metrics. https://doi.org/10.1101/327833

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

scRM: A single-cell foundation model trained on scRNA-seq reads

To our knowledge, scRM is the first single-cell foundation model trained on sequencing reads, demonstrating that raw data pretraining is not only feasible but competitive at a fraction of the computational cost. We anticipate that scaling this framework and exploring alternative architectures will unlock biological information inaccessible to count models and extend the dataset-specific pretraining it already makes possible. The source code is available at https://github.com/altos-labs/scRM.

bioinformatics↗
bioRxiv2026

Local fractal dimension as an indicator of functional reorganization in molecular interaction networks

Changes in protein abundance incompletely describe how molecular perturbations reshape cellular organization. Here we introduce DIFFRACT, a framework that integrates quantitative proteomics with protein-protein interaction networks to identify reorganization through changes in local fractal dimension. Molecular associations define effective distances between connected proteins, allowing the geometry of their surrounding network neighborhoods to be compared across conditions. Analyses across four organisms (E. coli, A. thaliana, C. elegans, and H. sapiens) and three interaction resources (STRING, BioGRID, and IntAct) establish heterogeneous local scaling, while controlled perturbations demonstrate sensitivity to regional changes and invariance to uniform distance rescaling. Applied to RAVER1-deficient A549 cells, DIFFRACT identifies components associated with established RNA-regulatory functions and candidate roles in tRNA biology, mitochondrial organization and ciliogenesis. Selected RNA-regulatory and ciliary themes are not significantly enriched by abundance-ranked gene set analysis, supporting complementary information from network geometry. Selected functional annotations also recur in HEK293T cells. Guided by the ciliary predictions, we show that RAVER1 depletion increases the proportion of ciliated A549 cells 2.57-fold and increases cilium length. DIFFRACT thus connects changes in molecular network geometry to experimentally testable biological responses, extending the interpretation of proteomic perturbations beyond differential abundance.

bioinformatics↗
bioRxiv2026

Peripheral Exercise Transcriptomes Do Not Converge on Glutamatergic or Neuroinflammatory Signatures: A Cross-Platform Meta-Analysis

Executive dysfunction following physical overtraining has been linked to glutamatergic dysregulation in the lateral prefrontal cortex (LPFC), but whether this central mechanism is reflected in peripheral gene expression is unknown. This study tested whether peripheral transcriptomic responses to exercise converge on the glutamatergic or neuroinflammatory pathways implicated in LPFC dysfunction. Three publicly available GEO datasets spanning skeletal muscle, whole blood, and circulating neutrophils were analyzed across four exercise groups using limma, with a discovery threshold of nominal p < 0.05 and absolute log2 fold change above 0.5. Differential expression analysis identified 7,311 unique differentially expressed genes (DEGs) across the four contrasts, but overlap between datasets was limited: 7 genes were shared between skeletal muscle and blood, 38 between skeletal muscle and the GSE43856 contrasts, and 90 between blood and the GSE43856 contrasts, with none significant across all three studies. Gene Ontology and KEGG enrichment of the DEG union identified ribonucleoprotein complex biogenesis, ribosome biogenesis, protein folding, and T cell activation as the dominant biological processes; no glutamatergic or neuroinflammatory terms reached significance. Cross-tissue concordance between skeletal muscle and blood was absent (r = 0). Effect sizes among the genes driving the top enriched pathways were dominated by ribosomal proteins, heat shock proteins, and translation initiation factors rather than glutamate receptors or transporters. These findings indicate that peripheral transcriptomic responses to exercise are tissue-specific and dominated by proteostasis, translational, and immune programs rather than glutamatergic signaling. Direct assessment of central nervous system tissue or fluid-based glutamatergic biomarkers may be required to link overtraining-induced fatigue to LPFC dysfunction.

bioinformatics↗