Using Machine Learning to Understand the Kinematics of Gesturing During Dual-Task Walking and Obstacle Crossing

Abstract

Study Rationale One out of four older adults experience a fall annually and over three million older adults are seen in emergency rooms due to a fall. Falls typically happen during walking in older adults and one out of five older adult falls occur on uneven ground. Gait involves both cognitive and motor demands and walking with extemporaneous speech is a cognitively demanding task, even for healthy young adults. When completing a dual-task with a complex locomotor task, such as obstacle avoidance, older adults have a greater risk of falling. Walking while talking (a dual-task (DT)), is an everyday task that can be used to understand the fall-risk in older adults due to the negative impacts in gait caused by increased cognitive load. Extemporaneous speech requires cognitive processes to retrieve lexical information, identify and select correct words, then words are uttered using complex motor processes. Hand gestures facilitate the lexical retrieval process during extemporaneous speech and frequently occur when someone is performing a DT, causing disruption in arm swing coordination. Arm swing is a cyclical process that occurs during gait where the arms swing out of phase relative to the legs, which helps maintain stability during gait. It remains unknown if gesturing during DT gait and obstacle crossing influences older adult stability by disrupting the coordination of typical arm-swing. The purpose of this study is: 1) to clinically examine the influence of gestures on fall-risk during DT walking and obstacle crossing, and 2) to examine the feasibility of machine learning models to recognize a hand gesture recorded with motion capture technology during a DT. The results from this study will reveal the relationship between gesturing and gait and could serve as indication of cognitive motor capacity and promote interventions to reduce fall-risk. Methods Twelve community-dwelling older adults were included in this analysis of a data from a previous study (women = 7, age = 70.3 years). Motion capture technology recorded performance as the participants walked and spoke extemporaneously about a randomly assigned topic (Vicon Nexus 2.x, Oxford UK), wearing 39 reflective markers according to the Vicon Plug-in-Gait full-body marker model. Participant performance in two conditions: 1) DT walking with extemporaneous speech and 2) obstructed DT walking with extemporaneous speech, were analyzed in this study. Plug-in Gait kinematics (i.e., ankle angles, wrist angles, elbow angles, etc.) were used as primary features for the machine learning model, and the central difference method was used to calculate the angular velocity and acceleration. Spatiotemporal outcomes (i.e., gait speed, step length, step width, and margin of stability) were calculated to determine the effect of gestures on dual-task walking. Approach distance, landing distance, and toe clearance were calculated during dual-task obstacle crossing, in addition to the previously mentioned spatiotemporal measures, to understand the effect of gestures on obstacle crossing. Interlimb coordination metrics (i.e., cross covariance coefficients, and continuous relative phase) were also calculated and compared between walking and obstacle crossing in older adults who gesture versus those who did not gesture. Gestures were then manually identified, by a researcher, from the start of the gesture to the end of the gesture. Gesture time was used to train a supervised machine learning model. One sliding window (20-frame window with 10-frame overlap) was tested to convert the joint angle, angular velocity, and angular acceleration data into the proper format for the machine learning algorithms. Data were then input into the KNN model. The machine learning model returned labeled time frames as a gesture or non-gesture, and this was compared to the manually identified gesture time frames. The resultant time frames indicated the comparison points for gesture versus non-gesture time and allowed us to compare gait performance across the two cognitive-motor states. We compared gait performance during gesture and non-gesture times between DT overground walking and obstacle avoidance. Expected Outcomes Results from this study will reveal the clinical relationship between gestures and gait, and the feasibility of using machine learning for recognizing gestures during DT walking. Once gestures can accurately be identified, gait changes due to the disruption in arm swing will be revealed for the first time. Using machine learning approaches to identify a gesture from motion capture data will enable reliable examination of the effect of gestures on DT gait in older adults, which may serve as an indicator of cognitive-motor function and fall-risk.