PROCare4Life

Besides my research work as a postdoctoral researcher at Maastricht University, I also work for the H2020 European project, PROCareLife. PROCare4Life is an e-health care platform that aims to provide an integrated e-health care monitoring system for seniors with neurodegenerative diseases. In PROCare4Life, my research aims to develop a personalized and dynamic multimodal fusion of behaviors based on data collected from patient symptoms, activities, cognitive abilities, and health records. The multimodal fusion integrates domain knowledge of socio-health professionals into Bayesian Networks to develop a personalized decision support system, improving patients’ quality of life by detecting deviations in their routines and health conditions. Please check the PROCare4Life website for more information. In this project, I also carried out administrative tasks, where I led the WP3 under the Supervision of Dr. Stelios Asteriadis between 2020 and 2021.

WP3 is responsible for the sensorial ecosystem of PROCare4life. Sensory tracking acquires data through sensors from wristbands, mobile phones, binary object sensors, and cameras. In particular, these sensors gather various raw measurements, such as heart rate, number of sleeping hours, number of steps, accelerometers, gyroscopes, and human-body trajectories. As part of the sensorial ecosystem of PROCare4Life, Sensory Data Tracking (SDT) was developed to enable the tracking and gathering of sensorial data from different users. Using sensory monitoring, PROCare4Life learns of users’ behavioral habits and detects disease-specific symptoms, e.g., freezing, festination, loss of balance, wandering, confusion, and falls. An essential component of PROCare4Life is the sensorial ecosystem, which provides the necessary tools and modules to track disease progression and facilitate communication between patients with neurodegenerative diseases and the corresponding social and health professionals.

In addition, I worked on the multimodal fusion task, where I developed a novel framework that focuses on improving the socio-health care of people with AD and PD. The framework uses human activity recognition algorithms to monitor patients’ physical activities, sleep patterns, and medication intake. It utilizes patients’ socio-demographic data and health records to provide personalized socio-health care. Subsequently, the proposed framework generates high-level representations of diseases’ symptoms, physical activities, sleep patterns, medication intake, cognitive states, comorbidities, and socio-demographic information, health records. The obtained representations are considered scores generated from analyzing long-term patterns of routines and symptoms. The paper presents the multimodal fusion approach of the proposed framework, which aims to utilize the modalities (scores) to detect deviations in patients’ regular routines. The multimodal fusion integrates the domain knowledge of clinical professionals into Bayesian networks, utilizing domain knowledge for the modeling and representation of data. The multimodal fusion generates probabilities of the detected deviations in the patient’s conditions concerning sleep, motor functions, physical activity, and cognitive abilities.

Please check the current output of the work on this project in the following items:

• Esam Ghaleb, Yusuf Can Semerci, and Stylianos Asteriadis. 2022. Modelling Behaviours of People Living with Neurodegenerative Conditions. In Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA ‘22). Association for Computing Machinery, New York, NY, USA, 351–357. https://doi.org/10.1145/3529190.3534721
• Esam Ghaleb & others. Personalized and Dynamic Multimodal Fusion of Behavioural Analysis of People Living with Neurodegenerative Conditions (work under progress)
• Blog Post: PROCare4Life Sensorial Ecosystem