A new family of statistical tests for responses in point-event and time-series data for one- and two-sample comparisons

Abstract

Quantifying whether and when signals are modulated by autonomous or external events is ubiquitous in the field of neuroscience. Existing statistical approaches, however, are not ideally suited to do this, especially when the signals under scrutiny show temporal autocorrelations. For example, a standard approach in the analysis of calcium imaging data is to use a t-test on predetermined time-windows to quantify whether neurons respond (differently) to an event of interest. While this is attractive because of its simplicity, only average signal differences can be detected. In practice, neurons often show complex response dynamics which are missed by conventional statistical tests. More advanced methods, such as bin-wise ANOVAs, do not share this drawback, but can suffer from high false-positive rates or "ghost correlations" when applied to temporally autocorrelated data. To solve this issue, we have developed three novel statistical tests extending the original ZETA-test to other use cases: 1) a test for time-series data; 2) a two-sample test to detect differences in neural responses between two conditions; and 3) a two-sample test to detect differences in time-series data between two conditions. In addition, we have improved upon the original ZETA-test. We show that our methods have a statistical sensitivity superior to t-tests and ANOVAs and work well with temporally autocorrelated data where other approaches fail. Our methods are widely applicable and we present example applications to Neuropixels data, two-photon GCaMP imaging data, and human electrocorticogram data. Open-source code for implementations in MATLAB and Python is available on GitHub and PyPi. Significance StatementNeurophysiology involves the detection of weak signals in a noisy background. Often these signals, as well as the background, are temporally autocorrelated. This means that many statistical methods, such as the ANOVA, are inappropriate. In some cases, these statistical tests produce "ghost" signals that mislead researchers into thinking there is an effect of their experiment, when in reality there is none. We have therefore developed a new family of statistical tests, the ZETA-tests, that are not negatively impacted by temporal autocorrelations. They do not use arbitrary (hyper)parameters, like bin size, and show superior statistical sensitivity compared to other methods like t-tests and ANOVAs. We provide easy-to-use implementations and expect our methods to be useful for many neuroscientists from various disciplines.