Environment Classification for Robotic Leg Prostheses and Exoskeletons using Deep Convolutional Neural Networks

Laschowski, B.; McNally, W.; Wong, A.; McPhee, J.

doi:10.1101/2021.06.24.449600

Home/Archive/bioRxiv

bioRxiv · 10.1101/2021.06.24.449600

Environment Classification for Robotic Leg Prostheses and Exoskeletons using Deep Convolutional Neural Networks

Laschowski, B.·McNally, W.·Wong, A.·McPhee, J.

View original source ↗Export metadata ↓

Abstract

Robotic leg prostheses and exoskeletons can provide powered locomotor assistance to older adults and/or persons with physical disabilities. However, the current locomotion mode recognition systems being developed for intelligent high-level control and decision-making use mechanical, inertial, and/or neuromuscular data, which inherently have limited prediction horizons (i.e., analogous to walking blindfolded). Inspired by the human vision-locomotor control system, we designed and evaluated an advanced environment classification system that uses computer vision and deep learning to forward predict the oncoming walking environments prior to physical interaction, therein allowing for more accurate and robust locomotion mode transitions. In this study, we first reviewed the development of the ExoNet database - the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a hierarchical labelling architecture. We then trained and tested over a dozen state-of-the-art deep convolutional neural networks (CNNs) on the ExoNet database for large-scale image classification of the walking environments, including: EfficientNetB0, InceptionV3, MobileNet, MobileNetV2, VGG16, VGG19, Xception, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169, and DenseNet201. Lastly, we quantitatively compared the benchmarked CNN architectures and their environment classification predictions using an operational metric called NetScore, which balances the image classification accuracy with the computational and memory storage requirements (i.e., important for onboard real-time inference). Although we designed this environment classification system to support the development of next-generation environment-adaptive locomotor control systems for robotic prostheses and exoskeletons, applications could extend to humanoids, autonomous legged robots, powered wheelchairs, and assistive devices for persons with visual impairments.

Source connections

Published article ↗

Explore related subjects

bioengineering Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Laschowski, B., McNally, W., Wong, A., McPhee, J.. 2021-06-25. Environment Classification for Robotic Leg Prostheses and Exoskeletons using Deep Convolutional Neural Networks. https://doi.org/10.1101/2021.06.24.449600

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Synergy-informed bilevel optimization reveals joint-specific benefits for realistic exoskeleton torque prediction

Simulation-driven optimization can guide exoskeleton control design, but its utility is limited by the accuracy with which muscle coordination is predicted. Strict effort-minimization formulations predict non-physiological muscle shutdown under high exoskeleton assistance, leading to peak assistive torques that exceed values reported in human-in-the-loop experiments. We evaluated the accuracy of synergy-informed simulations at predicting assistance across four single-joint exoskeleton configurations: ankle plantarflexion, knee extension, hip flexion, and hip abduction. We implemented a bilevel optimization to predict a parameterized assistive torque that minimizes metabolic cost, with assisted muscle dynamics estimated via either effort-minimization simulations (minimizing muscle effort only) or synergy-informed simulations (minimizing muscle effort constrained to a synergy structure). We used experimental data from five unimpaired adults walking at their preferred speed and compared results descriptively between the two formulations within each subject. Assistance based on effort-minimization simulations shut down agonist muscle activations and overestimated peak assistive torque compared with experimental values. In contrast, constraining muscle controls to a synergy structure prevented non-physiological muscle shutdown and yielded assistive torques closer to those measured in human-in-the-loop experiments. This benefit was not observed equally across all assisted degrees of freedom or subjects, emerging only where no single synergy could be suppressed to unload the assisted muscles. Synergy weights can therefore be interpreted as joint-specific predictors of when simulations can reliably inform wearable robot design.

bioengineering↗

bioRxiv2026

Engineered microvessel networks -- Longitudinal transcriptomic profiling validates chip-grown tissues as a robust model for studying vasculature

Vascularised Organ-on-Chip (OoC) models are promising New Approach Methodologies (NAMs), but their translation requires validation for defined Contexts of Use (CoU). Here, we combine microfluidic microvessel networks (MVNs) with longitudinal bulk and single-cell RNA sequencing to determine how cell provenance influences engineered vascular phenotype. Stromal cell identity produced distinct network architectures and transcriptional trajectories, demonstrating that support cells, commonly treated as interchangeable, represent an important CoU variable. Primary human pancreatic microvascular endothelial cells failed to form stable perfusable networks, but supplementation with 10% Human Umbilical Vein Endothelial Cells (HUVECs) or Blood Outgrowth Endothelial Cells (BOECs) rescued self-assembly. Single-cell transcriptomics identified specialised endothelial populations associated with vascular morphogenesis, including tip, stalk, phalanx, endothelial to mesenchymal transition (EndMT) and pericyte-like endothelial cells. Failed networks lacked coherent endothelial specialisation, whereas rescued networks restored stabilising cell states and revealed FLRT2--UNC5B signalling within pericyte-like HUVECs, recapitulating a pathway implicated in angiogenic development in vivo. These findings establish cell provenance as a critical determinant of microvascular NAM performance and demonstrate how transcriptomic validation can define and strengthen the biological relevance of microvessel-on-chip models for specific CoUs.

bioengineering↗

bioRxiv2026

Benchmarking foundation models for resting-state functional magnetic resonance imaging time series in cognitive aging research

Foundation models trained on large neuroimaging datasets could help smaller studies investigate cognition from brain recordings. However, whether the foundation model-derived representations improve assessment of current cognition, prediction of follow-up cognition, or characterization of cognitive change remains unclear. We evaluated nine configurations from five model families that process regional resting-state functional magnetic resonance imaging time series, using 3,103 scans from 1,653 participants across two observational and five intervention aging cohorts. Primary frozen representations showed no consistent superiority over conventional functional connectivity for concurrent cognitive associations or diagnostic classification. Changing the readout improved selected concurrent cognitive associations without updating model weights, indicating that performance depended partly on how representations were extracted. Additional post-training and embedding alignment yielded gains in selected settings but did not consistently improve performance across outcomes. In the primary longitudinal analyses, imaging representations did not establish added value beyond baseline clinical measures. These findings show that readout choice influences concurrent cognitive associations, while the added value of these representations for longitudinal outcomes remains unestablished.

bioengineering↗