Valection: Design Optimization for Validation and Verification Studies

Cooper, C. I.; Yao, D.; Sendorek, D. H.; Yamaguchi, T. N.; P'ng, C.; Houlahan, K. E.; Caloian, C.; Fraser, M.; SMC-DNA Challenge Participants,; Ellrott, K.; Margolin, A. A.; Bristow, R. G.; Stuart, J. M.; Boutros, P. C.

doi:10.1101/254839

bioRxiv · 10.1101/254839

Valection: Design Optimization for Validation and Verification Studies

Cooper, C. I.·Yao, D.·Sendorek, D. H.·Yamaguchi, T. N.·P'ng, C.·Houlahan, K. E.·Caloian, C.·Fraser, M.·SMC-DNA Challenge Participants,·Ellrott, K.·Margolin, A. A.·Bristow, R. G.·Stuart, J. M.·Boutros, P. C.

View original source ↗Export metadata ↓

Abstract

BackgroundPlatform-specific error profiles necessitate confirmatory studies where predictions made on data generated using one technology are additionally verified by processing the same samples on an orthogonal technology. In disciplines that rely heavily on high-throughput data generation, such as genomics, reducing the impact of false positive and false negative rates in results is a top priority. However, verifying all predictions can be costly and redundant, and testing a subset of findings is often used to estimate the true error profile. To determine how to create subsets of predictions for validation that maximize inference of global error profiles, we developed Valection, a software program that implements multiple strategies for the selection of verification candidates.\n\nResultsTo evaluate these selection strategies, we obtained 261 sets of somatic mutation calls from a single-nucleotide variant caller benchmarking challenge where 21 teams competed on whole-genome sequencing datasets of three computationally-simulated tumours. By using synthetic data, we had complete ground truth of the tumours mutations and, therefore, we were able to accurately determine how estimates from the selected subset of verification candidates compared to the complete prediction set. We found that selection strategy performance depends on several verification study characteristics. In particular the verification budget of the experiment (i.e. how many candidates can be selected) is shown to influence estimates.\n\nConclusionsThe Valection framework is flexible, allowing for the implementation of additional selection algorithms in the future. Its applicability extends to any discipline that relies on experimental verification and will benefit from the optimization of verification candidate selection.

Source connections

Published article ↗

Explore related subjects

bioinformatics Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Cooper, C. I., Yao, D., Sendorek, D. H., Yamaguchi, T. N., P'ng, C., Houlahan, K. E., Caloian, C., Fraser, M., SMC-DNA Challenge Participants,, Ellrott, K., Margolin, A. A., Bristow, R. G., Stuart, J. M., Boutros, P. C.. 2018-01-28. Valection: Design Optimization for Validation and Verification Studies. https://doi.org/10.1101/254839

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Cross-modal transfer enables zero-shot prediction of drug responses across cell types

Genetic perturbation responses capture the functional consequences of gene-level interventions, providing functional information for drug-response prediction beyond baseline gene expression. Within the same cellular context, genetic and chemical perturbations engage shared regulatory networks, providing a biological basis for transferring response information between perturbation modalities. Here we introduce GRAFT, a framework that combines genetic-to-chemical transfer with cross-context prediction to infer drug-induced transcriptional responses without drug measurements in the target context. Across benchmarked LINCS and external contexts, GRAFT improved response correlation, directional agreement and magnitude reconstruction over competing methods, particularly for drugs with context-dependent responses. GRAFT-Adapt further refines predictions of unmeasured responses using a small number of drug measurements from the target context. GRAFT-Atlas spans 13 cellular contexts and 7,856 drug catalog entries, comprising 102,128 predictions for drug-context combinations. The atlas captured recognizable pharmacological neighborhoods, shared functional programs, context-dependent transcriptional changes and structural-functional associations supported by molecular docking. Across ten iPSC lines from distinct donors, GRAFT identified candidate inter-donor differences, with functional support for selected candidates from measured genetic perturbation responses. Together, GRAFT enables perturbation-informed prediction and comparison of drug responses across cellular contexts.

bioinformatics↗

bioRxiv2026

Interpreting graph neural networks reveals microbial association patterns in inflammatory bowel diseases

Traditional machine learning models trained on gut microbiome abundance profiles can distinguish individuals with inflammatory bowel disease (IBD) from non-IBD individuals. However, their interpretation typically considers taxa in isolation, making it difficult to understand how associations between taxa contribute to predictions. Here, we inferred individual-specific microbial association networks from longitudinal metagenomics data of 106 individuals in HMP2/iHMP and used them to train and interpret graph neural network (GNN) models, which we compared with abundance-based models such as Random Forest (RF). Although RF achieved higher predictive performance (AUPRC ~0.59 vs 0.446), interpreting the GNNs revealed that their predictions relied on a densely connected module of co-abundant Bacteroidales species. Unlike the taxa prioritized by RF, these species were not differentially abundant between conditions. In an independent cohort, this module carried most of the predictive signal, and removing its associations reduced performance to near chance. These results show that coordinated association structure, rather than abundance shifts alone, is a feature of the IBD gut microbiome, and that interpreting GNNs can uncover such patterns that complement abundance-based analyses.

bioinformatics↗

bioRxiv2026

FateDriver: Weakly supervised dual-view graph learning for transition-specific prioritization of candidate regulators in cellular reprogramming

Cellular reprogramming supports regenerative medicine, yet the endogenous regulators of a specified source-to-target transition remain incompletely defined. Existing computational methods rank candidate regulators from network topology, trajectory dynamics, or cell-state labels, leaving unused the factors experimentally delivered to induce the transition. We present FateDriver, a weakly supervised dual-view graph neural network that jointly represents cells and genes from single-cell transcriptomes, treats the delivered factors as sparse gene-level labels, and ranks the remaining genes for a specified transition under a positive--unlabeled formulation. Across two OSKM datasets and one GMT dataset, FateDriver ranked first or second among six methods in recovering externally curated reference genes, exceeded a supervision-matched non-graph model despite weaker co-expression with the delivered factors, and achieved the highest recovery of protein targets of pluripotency-inducing chemical cocktails in both OSKM datasets. Holding either the source or the endpoint fixed yielded distinct candidate programs, including a DDX58--interferon axis associated with cells lacking endogenous pluripotency markers. Applying the same framework to candidate source--target pairs prioritized distinct candidate origins for two in vivo partially reprogrammed intestinal populations. FateDriver converts sparse experimental knowledge into transition-specific hypotheses for regulator prioritization from single-cell trajectories.

bioinformatics↗