PhD Thesis Final Defense to be held on December 23, 2019, at 11:00

Photo Credit: Georgia Chalvatzaki

The examination is open to anyone who wishes to attend (Teleteaching Room, Central Library of NTUA).

Thesis Title: Human-Centered Modelling for Assistive Robotics: Stochastic Estimation and Robot Learning in Decision Making

Abstract: Accurately tracking and analysing human gait constitutes an essential functionality for various robotic applications, such as smart robotic walkers aiming to provide assistance to patients with different mobility impairment, social robot companions, and many other. A context-aware robot control architecture needs constant knowl-edge of the user’s kinematic state to assess the patient’s gait status and adjust its movement properly to provide optimal assistance. Towards this end, this dissertation focuses on research topics related to monitoring human walking, and especially the movement of the user’s legs in a continuous and dynamic interaction with a robotic assistive mobility platform. More specifically, in the first part of this dissertation various tracking techniques have been applied for the human legs state estimation using Kalman Filter (KF) and K- means clustering. Further on methodologies using Particle Filters (PFs) based on Sequential Importance Resampling (SIR) have been investigated, to better cope with the nonlinear and non-Gaussian nature in stochastic gait dynamics. A new monitoring methodology was then developed using PFs for the human state estimation with probabilistic data association (PDA). In parallel, gait analysis methodologies have been developed with the extraction of spatio-temporal parameters, which are useful in the clinical diagnosis of various pathological gait classes. In the next phase of the dissertation, a novel human gait tracking approach is proposed, that uses two PFs and PDA with an Interacting Multiple Model (IMM) scheme for a real-time selection of the appropriate motion model according to the human gait analysis and the use of the Viterbi algorithm for an augmented human gait state estimation. The gait state estimates also interact with the IMM as a prior information that drives the Markov sampling process, while the PDA ensures that the legs of the same person are tracked in a coupled manner. The observation data are provided by a Laser Range finder (LRF) mounted on a robotic assistant walker. A detailed experimental validation is presented using ground truth data from a motion capture system, which was used in real experiments with elder subjects who presented various mobility impairments. The validation analysis considers the algorithm’s accuracy, robustness to occlusions and clutter, and the gait state classification success, subject to the effect of different number of samples used in the PFs. The results obtained on the real data of the elder subjects who have voluntarily participated in the experiments, show the efficacy of the proposed algorithm in providing accurate and robust augmented human gait estimates with a small number of particles, thus demonstrating the potential of the methodology to be used in a real-time application in the field of assistive or rehabilitation robotics. Continuing the research in the direction of human-centric estimation and learning, we have proposed and developed a novel method for analysing human gait stability from non-wearable sensors, namely for the users of the robotic mobility assistance platform. In particular, we have used deep neural networks and specifically, Long Short-Term Memory (LSTM) networks, a special kind of recurrent neural networks, in an encoder-decoder framework for predicting human walking stability as safe or risk-of-fall, using as input multimodal data from camera and laser in a human-centered estimation process. The final contribution of this work relies on studying and developing various control strategies for the robotic assistant, using human-centered information regarding the kinematic state of each user. The initial approach considered a strategy for the coupled human-robot motion control and the development of a kinematic control that considers the mobility classification of the user, in order to change certain control parameters regarding the separation distance and the relative bearing in the human-robot setting. Despite the fact that such a method performed well for straight line navigation scenarion, it was not easy to accomodate the disturbances occurring to the system during more complex movements, like turnings To this end, this dissertation is completed by the proposal, analysis and application of a novel framework for a human-centered robot motion adaptation. The goal is to achieve a coupled human-robot motion in a front-following setting as if the patient was pushing the rollator him/herself. To this end, we propose a novel approach using Model-based Reinforcement Learning (MBRL) for adapting the control policy of the robotic assistant. This approach encapsulates all previous work developed during this doctoral research on human tracking and gait analysis from RGB-D and laser streams into a human-in-the-loop decision-making strategy. We use LSTM networks for designing a Human Motion Intention Model (HuMIM) and a Coupling Parameters Forecast model, leveraging on the outcome of human gait analysis. An initial LSTM-based policy network was trained via Imitation Learning (IL) from human demonstrations in a Motion Capture setup. This policy is then fine-tuned with the MBRL framework using tracking data from real patients. A thorough evaluation analysis proves the efficiency of the MBRL approach as a user-adaptive human-centered decision-making strategy.

Supervisor: Tzafestas Costas, Associate Professor

PhD student: Georgia Chalvatzaki