Validation of two-dimensional video-based inference of finger kinematics with pose estimation

Abstract

Accurate capture finger of movements for biomechanical assessments has typically been achieved within laboratory environments through the use of physical markers attached to a participants hands. However, such requirements can narrow the broader adoption of movement tracking for kinematic assessment outside these laboratory settings, such as in the home. Thus, there is the need for markerless hand motion capture techniques that are easy to use and accurate enough to evaluate the complex movements of the human hand. Several recent studies have validated lower-limb kinematics obtained with a marker-free technique, OpenPose. This investigation examines the accuracy of OpenPose, when applied to images from single RGB cameras, against a gold standard marker-based optical motion capture system that is commonly used for hand kinematics estimation. Participants completed four single-handed activities with right and left hands, including hand abduction and adduction, radial walking, metacarpophalangeal (MCP) joint flexion, and thumb opposition. Accuracy of finger kinematics was assessed using the root mean square error. Mean total active flexion was compared using the Bland-Altman approach, and coefficient of determination of a linear regression. Results showed good agreement for abduction and adduction and thumb opposition activities. Lower agreement between the two methods was observed for radial walking (mean difference between the methods of 5.03{degrees}) and MCP flexion (mean difference of 6.82{degrees}) activities, due to occlusion. This investigation demonstrated that OpenPose, applied to videos captured with monocular cameras, can be used for markerless motion capture for finger tracking with an error below than 11{degrees} and on the order of that which is accepted clinically. Author summaryDecreased hand mobility may limit functionality, and its quantification is fundamental to assess underlying impairments. Optical motion capture technologies are the most accurate means by which to quantify hand motion. As this approach involves placing markers on the skin and recording hand movements using multiple cameras, there are limitations of physical space, time requirements, and financial implications. Therefore, the adoption of these practices is confined to laboratory settings. In clinical settings, goniometry is used to quantify hand range of motion (ROM), but this also involves lengthy processes and requires face-to-face assessments. Alternative solutions have been investigated to quantify hand mobility remotely and support home-based care interventions. However, none has been shown to be accurate enough to replace the gold-standard measurement of hand ROM in clinical settings. Recently, markerless technologies that leverage artificial intelligence have exhibited great potential for human movement analysis, but these studies have validated markerless tracking technologies for the lower limb only. We demonstrate that the validity of these models can be extended to capture hand mobility, making it also possible to assess hand function remotely.