Biology subjects

Hoye, T. T.

Publications and source records attributed to Hoye, T. T..

14 recordsLinked to original sources

bioRxiv2026

A Framework for Clustering and Zero-Shot Labeling of Ecological Images Using Foundation Models

The increasing automation of biodiversity monitoring is generating large volumes of ecological images requiring taxonomic identification, creating a substantial annotation bottleneck. Assigning species-level labels requires considerable time and expert knowledge, limiting scalability. We propose a framework for unsupervised clustering and zero-shot labeling of ecological images using foundation models. The framework extracts image embeddings from pretrained models and groups images into taxonomic clusters without task-specific training. Species labels are assigned through zero-shot text matching between image embeddings and prompted information representing taxa expected in the dataset. We investigate dimensionality reduction and six clustering algorithms for organizing fine-grained imagery. An uncertainty score combines clustering and labeling information to triage images into three confidence tiers, allowing experts to focus on uncertain or potentially incorrect assignments. The framework is evaluated on four bird and moth datasets, including two annotated camera-trap collections. The camera-trap datasets, per-image species-level accuracy reached 84% on DK-Moth and 83% on HUGO-Bench Aves. An interactive report presents clusters and zero-shot labels by confidence tier and prioritizes cases requiring expert review. The framework provides a training-free approach for organizing and labeling ecological image collections, reducing manual annotation while retaining expert oversight and demonstrating the potential of foundation models for scalable, semi-automated annotation.

ecology↗

bioRxiv2026

InsectDCT: A generalized pipeline for detection, taxonomic classification, and tracking of insects in camera-trap recordings

Automated monitoring of insect pollinators in natural environments with insect camera traps and trained deep learning algorithms provides novel data for insect ecological studies. However, efficient and accurate image recognition analysis of the recorded images or videos is challenging, particularly for images containing small insects against complex backgrounds with diverse vegetation communities. Even when insects can be detected in images, identifying their taxonomy remains difficult, particularly in footage with low image resolution, light conditions, and distances from the plants, and in cases where insects appear blurry or only partially visible. In this work, we present InsectDCT, an AI-based pipeline for automated detection, hierarchical classification, and tracking of insects in footage of natural vegetation tested in different environments. The InsectDCT pipeline consists of three levels: insect Detection and localization, hierarchical taxonomic Classification, and spatio-temporal Tracking. In the first stage, insects are detected in time-lapse images or video recordings using the You Only Look Once (YOLO11) object detection architecture. Detection performance is improved using motion-enhanced images, which improve robustness in cluttered and 3 dimensional environments. The detector is trained on an extensive dataset that contains more than 60,000 images collected using camera traps deployed across a wide range of plant families and floral habitats. In the second stage, detected insects are classified using a hierarchical taxonomy-aware classification framework that covers 80 taxonomic groups. Classification is performed at multiple taxonomic levels, including order, family, and genus/species, allowing coarse and fine-grained ecological analyzes while accounting for varying levels of visual ambiguity. In the third stage, a multi-object tracking module is applied to high temporal-resolution image sequences and video data to associate detections of the same individual across time. InsectDCT code and all datasets are made publicly available.

ecology↗

bioRxiv2024

Harmonising digitised herbarium data to enhance biodiversity knowledge: creating an updated checklist for the flora of Greenland.

International efforts to digitise herbarium specimens provide the building blocks for a global digital herbarium. However, taxonomic changes and errors can result in inconsistencies when amalgamating specimen metadata, that compromise the assignment of occurrence records to correct taxa, and the subsequent interpretation of patterns in biodiversity. We present a novel workflow to mass-curate digital specimens. By employing existing digital taxonomic backbones, we aggregate specimen names by their accepted name and flag remaining cases for manual review. We then validate names using site-specific floras, balancing automation with taxonomic expert-based curation. Applying our workflow to the vascular plants of Greenland, we harmonised 175,266 digitised herbarium specimens and observations from 92 data providers from the Global Biodiversity Information Facility (GBIF). The harmonised metacollection for the Greenland flora contains 780 plant species. Our workflow increases the number of species known from Greenland compared to other currently available species checklists and increases the mean number of occurrences per species by 42.6. Our workflow illustrates the integration required in order to create a global, universally accessible digital herbarium, and shows how previous obstacles to database curation can be overcome through a combination of automation and expert curation. From the specific perspective of the Greenland flora, our approach arrives at a new checklist of taxa, a new curated metacollection of occurrence data, and revised estimates of plant richness. The list of taxa and their prevalence allow a new basis for biodiversity assessment and conservation planning. Societal Impact StatementDigitising plant collections has allowed for data to be aggregated across multiple collections, forming a single harmonised resource of unprecedented scale. This resource is only accurate once the database names are assigned to one accepted name per species. We established a semi-automated workflow for processing plant name data, leveraging taxonomic backbones and employing taxonomic expertise at key stages. Applying our workflow to the flora of Greenland, we developed a curated checklist of 780 species, capturing greater species richness than previously published, while also curating 175,266 plant records. Our findings redefine our knowledge of Greenlandic plant diversity, while harmonising a vast digital collection for further research.

ecology↗

bioRxiv2024

Towards edge processing of images from insect camera traps

Insects represent nearly half of all known multicellular species but knowledge about them lacks behind most vertebrate species. In part for this reason, they are often neglected in biodiversity conservation policies and practice. Computer vision tools, such as insect camera traps, for automated monitoring have the potential to revolutionize insect study and conservation. To further advance insect camera trapping and the analysis of their image data, effective image processing pipelines are needed. In this paper, we present a flexible and fast processing pipeline designed to analyse these recordings by detecting, tracking and classifying nocturnal insects in a broad taxonomy of 15 insect classes and resolution of individual moth species. A classifier with anomaly detection is proposed to filter dark, blurred, or partially visible insects that will be uncertain to classify correctly. A simple track-by-detection algorithm is proposed to track classified insects by incorporating feature embeddings, distance and area cost. We evaluated the computational speed and power performance of different edge computing devices (Raspberry Pis and NVIDIA Jetson Nano) and compared various time-lapse strategies with tracking. The minimum difference was found for 2-minute time-lapse intervals compared to tracking with 0.5 frames per second, however, for insects with fewer than one detection per night, the Pearson correlation decreases. Shifting from tracking to time-lapse monitoring would reduce the amount of recorded images and be able to perform edge processing of images in real-time on a camera trap with Raspberry Pi. The Jetson Nano is the most energy-efficient solution, capable of real-time tracking at nearly 0.5 fps. Our processing pipeline was applied to more than 5.7 million images recorded at 0.5 frames per second from 12 light camera traps during two full seasons located in diverse habitats, including bogs, heaths and forests.

ecology↗

bioRxiv2024

A deep learning pipeline for time-lapse camera monitoring of floral environments and insect populations

Arthropods, including insects, represent the most diverse group and contribute significantly to animal biomass. Automatic monitoring of insects and other arthropods enables quick and efficient observation and management of ecologically and economically important targets such as pollinators, natural enemies, disease vectors, and agricultural pests. The integration of cameras and computer vision facilitates innovative monitoring approaches for agriculture, ecology, entomology, evolution, and biodiversity. However, studying insects and their interactions with flowers and vegetation in natural environments remains challenging, even with automated camera monitoring. This paper presents a comprehensive methodology to monitor abundance and diversity of arthropods in the wild and to quantify floral cover as a key resource. We apply the methods across more than 10 million images recorded over two years using 48 insect camera traps placed in three main habitat types. The cameras monitor arthropods, including insect visits, on a specific mix of Sedum plant species with white, yellow and red/pink colored of flowers. The proposed deep-learning pipeline estimates flower cover and detects and classifies arthropod taxa from time-lapse recordings. However, the flower cover serves only as an estimate to correlate insect activity with the flowering plants. Color and semantic segmentation with DeepLabv3 are combined to estimate the percent cover of flowers of different colors. Arthropod detection incorporates motion-informed enhanced images and object detection with You-Only-Look-Once (YOLO), followed by filtering stationary objects to minimize double counting of non-moving animals and erroneous background detections. This filtering approach has been demonstrated to significantly decrease the incidence of false positives, since arthropods, occur in less than 3% of the captured images. The final step involves grouping arthropods into 19 taxonomic classes. Seven state-of-the-art models were trained and validated, achieving F1-scores ranging from 0.81 to 0.89 in classification of arthropods. Among these, the final selected model, EfficientNetB4, achieved an 80% average precision on randomly selected samples when applied to the complete pipeline, which includes detection, filtering, and classification of arthropod images collected in 2021. As expected during the beginning and end of the season, reduced flower cover correlates with a noticeable drop in arthropod detections. The proposed method offers a cost-effective approach to monitoring diverse arthropod taxa and flower cover in natural environments using time-lapse camera recordings.

ecology↗

bioRxiv2023

Deep learning to extract the meteorological by-catch of wildlife cameras

Microclimate - proximal climatic variation at scales of meters and minutes - can exacerbate or mitigate the impacts of climate change on biodiversity. However, most microclimate studies are temperature-centric, and do not consider meteorological factors such as sunshine, hail and snow. Meanwhile, remote cameras have become a primary tool to monitor wild plants and animals, even at micro-scales, with deep learning tools to rapidly convert images into ecological data. However, deep learning applications for wildlife imagery have focused exclusively on living subjects. Here, we identify an overlooked opportunity to extract latent, yet ecologically relevant, meteorological information. We produce an annotated image dataset of micrometeorological conditions across 49 wildlife cameras in South Africas Maloti-Drakensberg and the Swiss Alps. We train ensemble deep learning models to classify conditions as overcast, sunshine, hail or snow. Our best model achieves 91.7% accuracy on test cameras not seen during training. Furthermore, we show how effective accuracy is raised to 96% by disregarding 14.1% of classifications where ensemble member models did not reach a consensus. Unlike previous studies we test model performance in remote and novel locations, distinguishing overcast and sunny conditions in Svalbard, Norway with 79.3% accuracy (93.9% consensus accuracy). Our model rapidly classifies sunshine, snow and hail in almost 2 million unlabeled images. Resulting micrometeorological data illustrated common seasonal patterns of summer hailstorms and autumn snowfalls across mountains in the northern and southern hemispheres. However, daily patterns of sunshine and shade diverged between sites, impacting daily temperature cycles. Crucially, we leverage micrometeorological data to demonstrate that (1) experimental warming using open-top chambers shortens early snow events in autumn, and (2) image-derived sunshine marginally outperforms sensor-derived temperature when predicting bumblebee foraging. These methods generate novel micrometeorological variables in synchrony with biological recordings, enabling new insights from an increasingly global network of wildlife cameras.

ecology↗

bioRxiv2023

Hierarchical Classification of Insects with Multitask Learning and Anomaly Detection

Cameras and computer vision are revolutionising the study of insects, creating new research opportunities within agriculture, epidemiology, evolution, ecology and monitoring of biodiversity. However, a major challenge is the diversity of insects and close resemblances of many species combined with computer vision are often not sufficient to classify large numbers of insect species, which sometimes cannot be identified at the species level. Here, we present an algorithm to hierarchically classify insects from images, leveraging a simple taxonomy to (1) classify specimens across multiple taxonomic ranks simultaneously, and (2) highlight the lowest rank at which a reliable classification can be reached. Specifically, we propose multitask learning, a loss function incorporating class dependency at each taxonomic rank, and anomaly detection based on outlier analysis for quantification of uncertainty. First, we compile a dataset of 41,731 images of insects, combining images from time-lapse monitoring of floral scenes with images from the Global Biodiversity Information Facility (GBIF). Second, we adapt state-of-the-art convolutional neural networks, ResNet and EfficientNet, for the hierarchical classification of insects belonging to three orders, five families and nine species. Third, we assess model generalization for 11 species unseen by the trained models. Here, anomaly detection is used to predict the higher rank of the species not present in the training set. We found that incorporating a simple taxonomy into our model increased accuracy at higher taxonomic ranks. As expected, our algorithm correctly classified new insect species at higher taxonomic ranks, while classification was uncertain at lower taxonomic ranks. Anomaly detection can effectively flag novel taxa that are visually distinct from species in the training data. However, five novel taxa were consistently mistaken for visually similar species in the training data. Above all, we have demonstrated a practical approach to hierarchical classification based on species taxonomy and uncertainty during automated in situ monitoring of live insects. Our method is simple and versatile and could be implemented to classify a wide range of insects as well as other organisms.

ecology↗

bioRxiv2022

Accurate detection and identification of insects from camera trap images with deep learning

Reported insect declines have dramatically increased the global demand for standardized insect monitoring data. Image-based monitoring can generate such data cost-efficiently and non-invasively. However, extracting ecological data from images is more challenging for insects than for vertebrates because of their small size and great diversity. Deep learning facilitates fast and accurate insect detection and identification, but the lack of training data for coveted deep learning models is a major obstacle for their application. We present a large annotated image dataset of functionally important insect taxa. The primary dataset consists of 29,960 annotated insects representing nine taxa including bees, hoverflies, butterflies and beetles across more than two million images recorded with ten time-lapse cameras mounted over flowers during the summer of 2019. The insect image dataset was extracted using an iterative approach: First, a preliminary detection model identified candidate insects. Second, candidate insects were manually screened by users of an online citizen science platform. Finally, all annotations were quality checked by experts. We used the dataset to train and compare the performance of selected You Only Look Once (YOLO) deep learning algorithms. We show that these models detect and classify small insects in complex scenes with unprecedented accuracy. The best performing YOLOv5 model consistently identifies nine dominant insect species that play important roles in pollination and pest control across Europe. The model reached an average precision of 92.7% and recall of 93.8 % in detection and classification across species. Importantly, when presented with uncommon or unclear insects not seen during training, our model detects 80% of individuals and usually interprets them as closely related species. This is a useful property to (1) detect rare insects for which training data are absent, and (2) generate new training data to correctly identify those insects in future. Our camera system, dataset and deep learning framework show promising results in non-destructive monitoring of insects. Furthermore, resulting data are useful to quantify phenology, abundance, and foraging behaviour of flower-visiting insects. Above all, this dataset represents a critical first benchmark for future development and evaluation of deep learning models for insect detection and identification.

ecology↗

bioRxiv2020

Deep learning and computer vision will transform entomology

ABSTRACTMost animal species on Earth are insects, and recent reports suggest that their abundance is in drastic decline. Although these reports come from a wide range of insect taxa and regions, the evidence to assess the extent of the phenomenon is still sparse. Insect populations are challenging to study and most monitoring methods are labour intensive and inefficient. Advances in computer vision and deep learning provide potential new solutions to this global challenge. Cameras and other sensors that can effectively, continuously, and non-invasively perform entomological observations throughout diurnal and seasonal cycles. The physical appearance of specimens can also be captured by automated imaging in the lab. When trained on these data, deep learning models can provide estimates of insect abundance, biomass, and diversity. Further, deep learning models can quantify variation in phenotypic traits, behaviour, and interactions. Here, we connect recent developments in deep learning and computer vision to the urgent demand for more cost-efficient monitoring of insects and other invertebrates. We present examples of sensor-based monitoring of insects. We show how deep learning tools can be applied to the big data outputs to derive ecological information and discuss the challenges that lie ahead for the implementation of such solutions in entomology. We identify four focal areas, which will facilitate this transformation: 1) Validation of image-based taxonomic identification, 2) generation of sufficient training data, 3) development of public, curated reference databases, and 4) solutions to integrate deep learning and molecular tools.Significance statement Insect populations are challenging to study, but computer vision and deep learning provide opportunities for continuous and non-invasive monitoring of biodiversity around the clock and over entire seasons. These tools can also facilitate the processing of samples in a laboratory setting. Automated imaging in particular can provide an effective way of identifying and counting specimens to measure abundance. We present examples of sensors and devices of relevance to entomology and show how deep learning tools can convert the big data streams into ecological information. We discuss the challenges that lie ahead and identify four focal areas to make deep learning and computer vision game changers for entomology.Competing Interest StatementThe authors have declared no competing interest.GlossaryBin pickingan industrial term for robots that pick up one of many objects randomly placed in a container.Convolutional Neural Network (CNN)a deep learning algorithm in the family of neural networks with serval different layers commonly applied for image recognition and classification. A CNN can be trained to recognize various objects and patterns in an image. There are four main different operations in a CNN: convolution, activation functions, sub sampling, and fully connected layer. During training the learnable parameters of each convolutional and fully connected layer are adjusted so the CNN is able to recognize different patterns of the training data and used for final image classification.Data augmentationa technique that can be used to artificially expand the size of a training dataset by creating modified images with objects of interest for classification.Machine learninga subset of artificial intelligence associated with creating algorithms that can change themselves without human intervention to get the desired result – by feeding themselves through structured data.Deep learninga subset of machine learning where algorithms are created and function similarly to machine learning, but where there are many levels of these algorithms, each providing a different interpretation of the data it conveys.DNA barcodingIdentification of a species using a short, standardised gene fragment.Initializationdescription of an object to be tracked.Training dataclassified images (e.g. images of known species identified by experts) that are recorded to train a deep learning model.Precisionthe number of true positives divided by the sum of true positives and false positivesRecallalso called the true positive rate, is the number of true positives divided by the sum of true positives and false negatives.Classification accuracythe sum of true positives and true negatives divided by the total number of specimens.View Full Text

ecology↗

bioRxiv2020

Earlier springs enable High-Arctic wolf spiders to produce a second clutch

Spiders at southern latitudes commonly produce multiple clutches, but this has not been observed at high latitudes where activity seasons are much shorter. Yet the timing of snowmelt is advancing in the Arctic, and may allow some species to produce an additional clutch. To determine if this is already happening, we used specimens of the wolf spider Pardosa glacialis caught by pitfall traps from the long-term (1996-2014) monitoring program at Zackenberg, Northeast Greenland. We dissected individual egg sacs and counted the number of eggs and partially developed juveniles, and measured carapace width of the mothers. Upon discovery of a bimodal frequency distribution of clutch sizes, as is typical for wolf spiders at lower latitudes producing a second clutch, we assigned egg sacs to being a first or second clutch depending on clutch size. We tested whether the median capture date differed among first and second clutches, whether clutch size was correlated to female size, and whether the proportion of second clutches produced within a season was related to climate. We found that assigned second clutches appeared significantly later in the season than first clutches. In years with earlier snowmelt, first clutches occurred earlier and the proportion of second clutches produced was larger. This result, likely a result of female spiders producing first clutches earlier in those years, which allowed time for another clutch. Clutch size for first clutches was correlated to female size, while this was not the case for second clutches. Our results provide the first evidence for Arctic invertebrates producing additional clutches in response to warming. This could be a common but overlooked phenomenon due to the challenges associated with long-term collection of life history data in the Arctic. Moreover, given that wolf spiders are a widely distributed, important tundra predator, we may expect to see population and food web consequences of their increased reproductive rates.

ecology↗

bioRxiv2020

Simple attributes predict the importance of plants as hosts to the richness of fungi and arthropods

Consumers constitute the vast majority of global terrestrial biodiversity. Yet, local consumer richness is poorly understood. Plant species richness offers a simple hypothesis to how the diversification of carbon substrates may promote the diversity of arthropods and fungi. We took this one step further and used databases on plant-consumer interaction links to derive the richness of associated biota per plant species (link score). Using a species inventory of 130 sites we investigated 1) how well the link score could be predicted by plant attributes and 2) if the sum of plant species observed or predicted link scores could predict site richness of arthropods and macrofungi better than plant species richness alone. We found plant link scores to be positively related to plant size, abundance, nativeness and ectomycorrhizal status. Link based indices generally improved prediction of richness, stressing the importance of plants as niche space for the megadiverse groups of insects and fungi.

ecology↗

bioRxiv2020

A light trap and computer vision system to detect and classify live moths (Lepidoptera) using tracking and deep learning

Insect monitoring methods are typically very time consuming and involves substantial investment in species identification following manual trapping in the field. Insect traps are often only serviced weekly resulting in low temporal resolution of the monitoring data, which hampers the ecological interpretation. This paper presents a portable computer vision system capable of attracting and detecting live insects. More specifically, the paper proposes detection and classification of species by recording images of live individuals attracted to a light trap. An Automated Moth Trap (AMT) with multiple light sources and a camera was designed to attract and monitor live insects during twilight and night hours. A computer vision algorithm referred to as Moth Classification and Counting (MCC), based on deep learning analysis of the captured images, tracked and counted the number of insects and identified moth species. Observations over 48 nights resulted in the capture of more than 250,000 images with an average of 5,675 images per night. A customized convolutional neural network was trained on 2,000 labelled images of live moths represented by eight different species, achieving a high validation F1-score of 0.93. The algorithm measured an average classification and tracking F1-score of 0.71 and a tracking detection rate of 0.79. Overall, the proposed computer vision system and algorithm showed promising results as a low-cost solution for non-destructive and automatic monitoring of moths.

ecology↗

bioRxiv2019

Multi-taxon inventory reveals highly consistent biodiversity responses to ecospace variation

Amidst the global biodiversity crisis, identifying drivers of biodiversity variation remains a key challenge. Scientific consensus is limited to a few macroecological rules, such as species richness increasing with area, which provide limited guidance for conservation. In fact, few agreed ecological principles apply at the scale of sites or reserve management, partly because most community-level studies are restricted to single habitat types and species groups. We used the recently proposed ecospace framework and a comprehensive data set for aggregating environmental variation to predict multi-taxon diversity. We studied richness of plants, fungi, and arthropods in 130 sites representing the major terrestrial habitat types in Denmark. We found the abiotic environment (ecospace position) to be pivotal for the richness of primary producers (vascular plants, mosses, and lichens) and, more surprisingly, little support for ecospace continuity as a driver. A peak in richness at intermediate productivity adds new empirical evidence to a long-standing debate over biodiversity responses to productivity. Finally, we discovered a dominant and positive response of fungi and insect richness to organic matter accumulation and diversification (ecospace expansion). Two simple models of producer and consumer richness accounted for 77 % of the variation in multi-taxon species richness suggesting a significant potential for generalization beyond individual species responses. Our study widens the traditional conservation focus on vegetation and vertebrate populations unravelling the importance of diversification of carbon resources for diverse heterotrophs, such as fungi and insects.

ecology↗

bioRxiv2018

Vascular plants are strong predictors of multi-taxon species richness

Plants regulate soils and microclimate, provide substrate for heterotrophic taxa, are easy to observe and identify and have a stable taxonomy, which strongly justifies the use of plants as bioindicators in monitoring and conservation. However, insects and fungi make up the vast majority of species. Surprisingly, it remains untested whether plants are strong predictors of total multi-taxon species richness. To answer this question, we collected an extensive data set on species richness of vascular plants, bryophytes, macrofungi, lichens, plant-galling arthropods, gastropods, spiders, carabid beetles, hoverflies and OTU richness from environmental DNA metabarcoding. Plant species richness per se was a moderate predictor of richness of other taxa. Taking an ecospace approach to modelling, the addition of plant-derived bioindicators revealed 1) a consistently positive effect of plant richness on other taxa, 2) prediction of 12-55% of variation in other taxa and 48 % of variation in the total species richness.

ecology↗