A carregar...
5 resultados
Resultados da pesquisa
A mostrar 1 - 5 de 5
- Exploring Human Action Recognition for Rehabilitation Game ApplicationPublication . Lopes, Júlio Castro; Van-Deste, Isaac; Lopes, Rui PedroThrough computer vision algorithms motion can be computed, which can be a crucial element to be integrated with serious game environments. To evaluate the efficacy of motion detection algorithms, the information of these algorithms can be used to perform Human Action Recognition (HAR). There are several algorithms to perform HAR, although skeleton approaches can be seen as the best way to isolate human motion. To extract the human skeleton representation, the work described in this paper evaluates three distinct methods: OpenPose (2D), YOLO-Pose (2D) and BlazePose (3D). The information translated by the skeleton representations is normalized by lightweight normalization algorithms (for further real-time application). To classify the video sequence and further action identification, a Long Short Term Memory network (LSTM), was used. Using the N-UCLA dataset, the highest F1 score of 0.745 was achieved using OpenPose skeleton extraction (2D), followed by the computation of the angles in each joint, demonstrating that the OpenPose skeleton representation can be the most viable solution for computing human motion in serious games.
- An architecture for capturing and synchronizing heart rate and body motion for stress inferencePublication . Lopes, Júlio Castro; Vieira, João; Van-Deste, Isaac; Lopes, Rui PedroThis paper aims to propose a system for capturing and synchronizing human heart rate (HR) and body motion (BMR) for stress inference. For this purpose, OpenPose skeletonbased method was used, which is capable of analyzing sequential videos, processing them frame by frame, and obtaining an approximation to the human figure composed of 18 key points, roughly corresponding to the joints. It is expected that by combining these two distinct measurements, HR and BMR, a more grounded evaluation of player stress levels while playing a Virtual Reality (VR) game, will be achieved. The experiment was conducted with 5 participants playing 5 different types of games, with different levels of intensity. During the game, the players wore a smartwatch to measure the HR and images were captured to calculate the BMR. Future work will assess this dataset to confirm the stress level in these 5 situations.
- Stress inference in a virtual reality game for rehabilitation with body motion and heart ratePublication . Lopes, Júlio Castro; Van-Deste, Isaac; Vieira, João; Lopes, Rui PedroThis paper proposes an architecture for detecting stress in a player, while playing a Virtual Reality (VR) game, by analyzing the player’s movements as well as the player’s Heart Rate (HR). For this effect, only a camera to analyze players’ Body Motion Rate (BMR) and a smartwatch to capture the HR, were used. As part of the computation of the BMR, computer vision techniques were used to detect the player’s skeleton, computing the difference between frames. A dataset was captured in this paper, while the players tested 5 different scenarios to induce different stress situations. The proposed dataset serves as a proof of concept to validate the relation between HR and BMR. Future work should investigate synthetic data generation techniques to improve dataset diversity and adaptability for Dynamic Difficulty Adjustment (DDA) systems. This research contributes to advancing stress detection in VR, with potential applications in rehabilitation, particularly for conditions such as schizophrenia, promoting improved well-being and stress management in the long run.
- Event-driven management system for smart pill dispensersPublication . Van-Deste, Isaac; Costa, Bruno A.; Lopes, Rui Pedro; Pereira, Ana I.Non-adherence on medication is increasing and there are several problems associated with it. There are many challenges in order to have full adherence to the prescribed treatments. Smart pill dispensers are being a new way to support patients who have complex prescriptions. However, those dispensers do not have much capacity and are very simple machines with few components. This paper presents a new architecture for smart pills dispensers based on Events-Driven, a distributed architecture where all the components have weak associations with each other. This architecture proves to be very advantageous for smart pill dispensers, increasing the flexibility and the resilience of these machines.
- Integration of heterogeneous components for dynamic adaptation of rehabilitation simulationsPublication . Van-Deste, Isaac; Lopes, Rui PedroThis dissertation investigates the integration of digital health technologies, artificial intelligence, and immersive environments to design a rehabilitation system for individuals who suffers from schizophrenia disease. Schizophrenia is a multifaceted complex mental disorder, that often leads to loss of physical and cognitive skills, reducing social interaction, with a high relapse rates, that turns rehabilitation a long-term challenge. Virtual Reality has been emerging as a promising tool for rehabilitation, sice it provides a safe, controlled, and motivating therapeutic scenarios. Upon this, the present work introduces a VR-based serious game capable of dynamically adapting its difficulty in response to physiological and behavioural signals. To achieve this solution, several different data from the patient, such as heart rate, body motion rate, and facial expression recognition were continuously monitored and processed to infer the patient’s stress levels in real time. The stress estimate is then integrated into a Dynamic Difficulty Adjustment mechanism, formulated in a Reinforcement Learning algorithm. A Deep Q-Network agent was trained within a simulated environment to learn optimal policies that balance challenge and engagement, ensuring that rehabilitation sessions remain neither frustrating nor trivial. Then, a distributed, event-driven architecture was designed to support the integration of heterogeneous modules, including the Virtual Reality application, a web dashboard, the dynamic difficulty system and a back office server, by using publish-subscribe communication, with Apache ActiveMQ Artemis, and containerized deployment, using Docker, to ensure scalability, modularity, and real-time interoperability. Experimental validation showed that the agent successfully learned to maintain stable levels of difficulty and stress, confirming the feasibility of adaptive rehabilitation driven by physiological signals. Overall, this dissertation demonstrates how the convergence of VR, distributed systems, and artificial intelligence can be harnessed to create adaptive rehabilitation tools that are not only technically robust but also clinically meaningful. The framework provides a foundation for future clinical studies, offering potential applications beyond schizophrenia, including in neurological rehabilitation, stress management, and personalized digital health therapies.