The Art of Not Knowing: Accommodating Structured Missingness in Biomedical Research

Abstract

Missing data remain a ubiquitous and critical challenge in large-scale clinical studies. Despite advances in imputation, most existing methods fail to address structured missingness, where data are missing according a deterministic pattern and which arise due to systematic patterns introduced by experimental design, site protocols, or cohort differences. These patterns violate key assumptions of most imputation algorithms, yet their impact is rarely evaluated. We demonstrate that structured missingness is a fundamental challenge to drawing valid inferences from standard imputation techniques. First, we present a comprehensive framework for understanding and accommodating the effects of structured missingness. Next, we show through simulations and real-world psychometric data with structured missingness, that widely used algorithms optimised for numerical precision (e.g., Extra Trees, AutoComplete) underperform due to site effects, while donor-based methods (e.g., MICE, hierarchical MICE) better preserve multivariate structure. We propose a novel hierarchical approach that provides optimal performance in simulated and experimental data. Finally, we show that commonly used accuracy metrics, such as mean squared error can obscure these failures, and are therefore inadequate for the evaluation of structured missingness. In contrast, other divergence-based metrics offer a more sensitive and interpretable alternative. We apply this approach to harmonising psychometric data across cohorts, which provides excellent item-level alignment across different instruments. Our study highlights the need for a paradigm shift in handling missing data within biomedical research, moving beyond conventional imputation frameworks to develop tools that can account for structured missingness. This shift is essential for ensuring reliable inference in multi-site clinical studies, precision medicine, and large-scale population analyses.