Biology subjects

Karstoft, H.

Publications and source records attributed to Karstoft, H..

3 recordsLinked to original sources

bioRxiv2024

Towards edge processing of images from insect camera traps

Insects represent nearly half of all known multicellular species but knowledge about them lacks behind most vertebrate species. In part for this reason, they are often neglected in biodiversity conservation policies and practice. Computer vision tools, such as insect camera traps, for automated monitoring have the potential to revolutionize insect study and conservation. To further advance insect camera trapping and the analysis of their image data, effective image processing pipelines are needed. In this paper, we present a flexible and fast processing pipeline designed to analyse these recordings by detecting, tracking and classifying nocturnal insects in a broad taxonomy of 15 insect classes and resolution of individual moth species. A classifier with anomaly detection is proposed to filter dark, blurred, or partially visible insects that will be uncertain to classify correctly. A simple track-by-detection algorithm is proposed to track classified insects by incorporating feature embeddings, distance and area cost. We evaluated the computational speed and power performance of different edge computing devices (Raspberry Pis and NVIDIA Jetson Nano) and compared various time-lapse strategies with tracking. The minimum difference was found for 2-minute time-lapse intervals compared to tracking with 0.5 frames per second, however, for insects with fewer than one detection per night, the Pearson correlation decreases. Shifting from tracking to time-lapse monitoring would reduce the amount of recorded images and be able to perform edge processing of images in real-time on a camera trap with Raspberry Pi. The Jetson Nano is the most energy-efficient solution, capable of real-time tracking at nearly 0.5 fps. Our processing pipeline was applied to more than 5.7 million images recorded at 0.5 frames per second from 12 light camera traps during two full seasons located in diverse habitats, including bogs, heaths and forests.

ecology↗

bioRxiv2024

A deep learning pipeline for time-lapse camera monitoring of floral environments and insect populations

Arthropods, including insects, represent the most diverse group and contribute significantly to animal biomass. Automatic monitoring of insects and other arthropods enables quick and efficient observation and management of ecologically and economically important targets such as pollinators, natural enemies, disease vectors, and agricultural pests. The integration of cameras and computer vision facilitates innovative monitoring approaches for agriculture, ecology, entomology, evolution, and biodiversity. However, studying insects and their interactions with flowers and vegetation in natural environments remains challenging, even with automated camera monitoring. This paper presents a comprehensive methodology to monitor abundance and diversity of arthropods in the wild and to quantify floral cover as a key resource. We apply the methods across more than 10 million images recorded over two years using 48 insect camera traps placed in three main habitat types. The cameras monitor arthropods, including insect visits, on a specific mix of Sedum plant species with white, yellow and red/pink colored of flowers. The proposed deep-learning pipeline estimates flower cover and detects and classifies arthropod taxa from time-lapse recordings. However, the flower cover serves only as an estimate to correlate insect activity with the flowering plants. Color and semantic segmentation with DeepLabv3 are combined to estimate the percent cover of flowers of different colors. Arthropod detection incorporates motion-informed enhanced images and object detection with You-Only-Look-Once (YOLO), followed by filtering stationary objects to minimize double counting of non-moving animals and erroneous background detections. This filtering approach has been demonstrated to significantly decrease the incidence of false positives, since arthropods, occur in less than 3% of the captured images. The final step involves grouping arthropods into 19 taxonomic classes. Seven state-of-the-art models were trained and validated, achieving F1-scores ranging from 0.81 to 0.89 in classification of arthropods. Among these, the final selected model, EfficientNetB4, achieved an 80% average precision on randomly selected samples when applied to the complete pipeline, which includes detection, filtering, and classification of arthropod images collected in 2021. As expected during the beginning and end of the season, reduced flower cover correlates with a noticeable drop in arthropod detections. The proposed method offers a cost-effective approach to monitoring diverse arthropod taxa and flower cover in natural environments using time-lapse camera recordings.

ecology↗

bioRxiv2023

Hierarchical Classification of Insects with Multitask Learning and Anomaly Detection

Cameras and computer vision are revolutionising the study of insects, creating new research opportunities within agriculture, epidemiology, evolution, ecology and monitoring of biodiversity. However, a major challenge is the diversity of insects and close resemblances of many species combined with computer vision are often not sufficient to classify large numbers of insect species, which sometimes cannot be identified at the species level. Here, we present an algorithm to hierarchically classify insects from images, leveraging a simple taxonomy to (1) classify specimens across multiple taxonomic ranks simultaneously, and (2) highlight the lowest rank at which a reliable classification can be reached. Specifically, we propose multitask learning, a loss function incorporating class dependency at each taxonomic rank, and anomaly detection based on outlier analysis for quantification of uncertainty. First, we compile a dataset of 41,731 images of insects, combining images from time-lapse monitoring of floral scenes with images from the Global Biodiversity Information Facility (GBIF). Second, we adapt state-of-the-art convolutional neural networks, ResNet and EfficientNet, for the hierarchical classification of insects belonging to three orders, five families and nine species. Third, we assess model generalization for 11 species unseen by the trained models. Here, anomaly detection is used to predict the higher rank of the species not present in the training set. We found that incorporating a simple taxonomy into our model increased accuracy at higher taxonomic ranks. As expected, our algorithm correctly classified new insect species at higher taxonomic ranks, while classification was uncertain at lower taxonomic ranks. Anomaly detection can effectively flag novel taxa that are visually distinct from species in the training data. However, five novel taxa were consistently mistaken for visually similar species in the training data. Above all, we have demonstrated a practical approach to hierarchical classification based on species taxonomy and uncertainty during automated in situ monitoring of live insects. Our method is simple and versatile and could be implemented to classify a wide range of insects as well as other organisms.

ecology↗