Empowering Cognitive Stimulation Therapy with Socially Assistive Robotics and EmoAion Recognition
Abstract: Socially Assistive Robotics (SAR) has already been widely used in mental health service and research, primarily among children with Autism Spectrum Disorder (tSD) and among older adults with dementia. Motivated by the benefits offered by SAR in mental health service and research, we envision that SAR can also benefit cognitive rehabilitation of individuals struggling with a wide range of mental health concerns, including adults with intellectual developmental disorders (IDD), people with neurodegenerative disorders such as Alzheimer’s disease etc. Cognitive rehabilitation involves guided practice on a range of standard tasks related to one or more cognitive domains. Such gain in cognition will increase autonomy among these individuals which in turn can improve quality of life and hence well-being of these individuals. Motivated by lack of adequate resources for providing support to individuals with mental health concerns, the benefits offered by cognitive rehabilitation and SAR in mental health service and research, we envision that SAR empower:d cognitive rehabilitation can positively affect the well-being of a wide variety of users.
In this thesis, we investigated the benefits of robot assisted cognitive rehabilitation for individuals with IDD. In the first part, we evaluated the fitness of robot assisted interventions for cognitive rehabilitation of individuals with ID. We conducted interviews with seven expert psychologists and professional caregivers working with individuals with ID. These interviews helped us to identify key aspects of a beneficial robot-assisted mental health intervention. To assess the impact of such robot-assisted mental health interventions on the users, we conducted a case study of robot interactions among six individuals with ID using NAO robot in different categories of interaction. We also compared the response of robotic interactions with non-robotic visual stimulations caused by a tactile gaming console. The results reported positive effects of robot interventions on the users and that the stimulations caused by tactile gaming console can significantly serve as complementary tool for therapeutic benefit of patients. We further evaluated the impact of robot-assisted mental health interventions on caregivers in multi-center trials. The results of the research confirmed a significant reduction in caregivers burden during execution of cognitive stimulation interventions for individuals with IDD and raised a concern about the need of a specific training of the caregivers to take maximum advantage of SAR in health care.
The second part concerns itself with empowering the social robots with automated and online emotion recognition ability for emotional adaptation in a bid to improve rehabilitation. We conducted a series of cognitive stimulation sessions among individuals with ID in a nearly real world settings to obtain a first ever annotated multimodal dataset (MuDERI) of individuals with IDD. MuDERI is an annotated multimodal dataset of audiovisual recordings, RGB-D videos and Electro-dermal activity (EDA), Electroencephalogram (EEG) physiological signals of 12 participants in actual settings, which were recorded as participants were elicited using personalized cognitive stimulation sessions. The dataset is publicly available. We further proposed an efficient wavelet-based method for artifacts attenuation of EDA signal during the online collection and analysis, while minimizing distortiens, using a stationary wavelet transform (SWT). The proposed method was tested on EDA recordings from publicly available driver dataset collected during real-world driving, and containing a high number of motion artifacts, and the results were compared to those of three state-of-the-art methods for EDA signal filtering. In addition, the proposed method was tested for the online filtering of EDA signals collected while twelve volunteers conducted tasks designed to elicit various stress states. The results evidenced that the prediction of arousal states can be significantly improved after motion artifacts removal, and that the proposed method outperforms existing approaches and it has a lower computational cost. Taken together, these results evidence the effectiveness of the proposed method for online EDA filtering in real world scenarios. Further, we reviewed feature extraction methods for emotion recognition fcom EDA based on 25 studies. We compared these features for feature selection using machine learning techniques on a publicly available AMIGOS dataset. We present the results of the performance of three feature selection methods and usage of selected feature types across time, frequency, timefrequency domains. We did not find any statistical evidence that any of the three empleyed feature solection methods outperform the others. However, the subject-dependent classification rosults were significantly higher than the subject-independent classification for both the arousal and the valence rerognition. MFCC and related statistical features were explored for the first time for the emotion recognition from EDA signals and they outperformed all other feature types, including the most commonly used SCR related features. We also compared our results with methods employed by researchers of AMIGOS dataset for classification of emotional states and they show that the EDA features explored in this study provided better performance, validating the findings of our study.