bioRxiv Science⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2023.09.30.560270

Positive Transfer of the Whisper Speech Transformer to Human and Animal Voice Activity Detection

Gu, N.·Lee, K.·Basha, M.·Ram, S. K.·You, G.·Hahnloser, R.
View original source ↗Export metadata ↓

Abstract

This paper introduces WhisperSeg, utilizing the Whisper Transformer pre-trained for Automatic Speech Recognition (ASR) for human and animal Voice Activity Detection (VAD). Contrary to traditional methods that detect human voice or animal vocalizations from a short audio frame and rely on careful threshold selection, WhisperSeg processes entire spectrograms of long audio and generates plain text representations of onset, offset, and type of voice activity. Processing a longer audio context with a larger network greatly improves detection accuracy from few labeled examples. We further demonstrate a positive transfer of detection performance to new animal species, making our approach viable in the data-scarce multi-species setting.1

Explore related subjects

animal behavior and cognitionPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Gu, N., Lee, K., Basha, M., Ram, S. K., You, G., Hahnloser, R.. 2023-10-02. Positive Transfer of the Whisper Speech Transformer to Human and Animal Voice Activity Detection. https://doi.org/10.1101/2023.09.30.560270

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Automated pup-level analysis reveals distinct effects of prenatal CBD and THC exposure on maternal retrieval

This study examined how prenatal CBD and {Delta}9-tetrahydrocannabinol (THC) exposure affects maternal caregiving in C57BL/6J mice. We developed the Machine-Automated Scoring of the Pup Retrieval Test (MAS-PRT) to overcome limitations of manual behavioral scoring. MAS-PRT integrates Multi-Animal DeepLabCut for dam and pup tracking with Detectron2 for dynamic nest reconstruction. Validated against 170 manually annotated retrieval trials, the pipeline showed high concordance with manual measurements and enabled reproducible extraction of encounter latency, retrieval latency, and locomotor trajectories. Prenatal exposure did not impair general nest-building or overall home-cage maternal care. However, pups from both CBD and THC groups showed reduced body weight at postnatal day 5. Cox proportional hazards modeling revealed divergent effects by compound: CBD-exposed dams exhibited a weight-dependent increase in probability of encountering and retrieving lighter pups, independent of pup sex. Spatial tracking further showed that dams traversed significantly shorter total trajectories when retrieving female progeny exposed to either compound. Longitudinal trial-by-trial analysis indicated intact task acquisition in controls and CBD dams, whereas THC dams displayed a flattened learning curve driven by lower retrieval latencies on initial trials. Together, these findings indicate that prenatal cannabinoid exposure does not produce generalized disruption of maternal care but instead induces compound-specific alterations in maternal reactivity, retrieval kinematics, and learning dynamics.

animal behavior and cognition↗
bioRxiv2026

A comparison of female competitive traits: Female aggression peaks at nest building but female song spans multiple contexts in a temperate songbird

Female-female competition is increasingly recognized as a key driver of female ornamentation, including birdsong, which often functions in intrasexual competition. However, the specific resources females use elaborate traits to compete for remain unclear. In addition, few studies have simultaneously investigated the use of multiple competitive traits in females, despite growing independent interest in these traits (e.g., female song and aggression). We investigated the competitive contexts of female song, aggression and calling behavior in northern house wrens (Troglodytes aedon) to determine which resources females compete for across the breeding season. We simulated conspecific territorial intrusions using female song at three breeding stages representing different contexts: arrival (mate and territory acquisition), nest building (nest site and breeding status defense), and egg laying (brood defense). We tested whether female song and physical aggression varied as reproductive resources shifted across the breeding cycle. Females were significantly more aggressive during nest building, showing 5.8 times greater odds of a higher-intensity aggressive response during nest building compared to arrival. Female song output was similar across early stages but declined during egg laying, though this was not statistically significant after correction for multiple comparisons and individuals varied substantially in overall singing propensity. Non-song vocalizations varied by call type and breeding stage. Calls associated with aggression occurred most frequently during nest building, consistent with peak physical aggression responses. Together, these results identify nest building as the stage of highest female aggression, consistent with heightened competition over nest cavities and associated breeding status in this cavity-nesting species. In contrast, female song occurred across all stages and appears to function in multiple competitive contexts. This study provides evidence for context and mode-specific female signaling in a temperate songbird and highlights that females strategically use aggression, calls, and song to mediate social conflict across breeding contexts.

animal behavior and cognition↗
bioRxiv2026

Tracking human foragers and their prey reveals adaptive predator-prey dynamics

Hunting for mobile prey is thought to have played a key role in hominin evolution, by providing high-quality nutrition that supported the development of the exceptionally large human brain. However, human-prey dynamics remain poorly understood because studies have not yet tracked human foragers and their prey simultaneously. Here, we employ high resolution tracking of groups of human foragers (ice-fishers) and their prey (fish shoals) to study human-prey dynamics. Our results show that foragers adaptively combined personal and social information in deciding where to forage and for how long, closely matching the prey distribution. Prey responded dynamically to human exploitation, showing increased attraction to fishing activity, alongside decreased biting probability. Furthermore, we found that foragers adaptively relied on memory, preferentially returning to areas with high prey presence, particularly when their current return rate was low. Our results show how human foragers overcome the challenges of extracting invisible, mobile and reactive prey by tightly tuning patch-selection, patch-leaving and patch-return decisions to the distribution and behaviour of their prey.

animal behavior and cognition↗