Thesis title: Content Generalization and Automated Testing through Intelligent Agents in Dynamic Serious Games for Health

Abstract: This doctoral thesis designs and implements a comprehensive framework for managing, generalizing, and testing content in Serious Games for Health (SGHs), incorporating artificial intelligence techniques with the aim of personalizing and enhancing the intervention. The development of SGHs is a time-consuming and costly process, while the lack of standardized methods for content reuse and evaluation of their mechanisms limits the ability to create effective SGHs. The aim of this thesis is to develop a framework that, independently of the content of the intervention, facilitates the faster implementation of SGHs, as well as the objective evaluation of the techniques used to produce personalized content in an automated manner. The proposed framework allows the content to be decoupled from the game mechanics, facilitating its reuse and adaptation to different health fields. It consists of three main axes: the generalization of the content of SGHs, the development of Procedural Content Generation (PCG) techniques, and the creation of a methodological framework for evaluating both the content of SGHs and the PCG techniques applied to them. The axes of the proposed framework are documented using a series of five original SGHs, which concern Obstructive Sleep Apnea (OSA), nutritional education, stress management, type 1 diabetes mellitus (T1DM), and obesity.

The first axis of the proposed framework concerns the generalization of the content of SGHs by disconnecting it from game mechanisms. This process initially involves the detailed recording of the elements of the SGH, including its scope, objectives, mechanisms, and graphics. These elements are then redesigned, removing references to the specific field and replacing them with a color coding system. The game mechanisms are generalized and adapted to function independently of the content, while the graphics related to the field are replaced by abstract representations that retain their basic characteristics. The final step involves the implementation of a "Content Manager," which provides programming interfaces for switching and customizing content. This approach allows the same SGH to be reused in different health fields, replacing only the content and not the game mechanics, achieving significant time and resource savings when developing new interventions. The effectiveness of this approach is evaluated by the transition of the SGH from the field of OSA to the field of depression, maintaining the same game mechanics but with completely different content that reflects the particularities of the new field.

The second axis of the proposed framework focuses on the development and generalization of PCG techniques for SGHs. A central element is the development of a PCG technique based on a genetic algorithm, which is initially applied to the SGH for OSA to create game character profiles and then extended to the SGH for T1DM and childhood obesity management to provide personalized recommendations and game missions. The process of generalizing the PCG technique involves defining the affected content, classifying it into categories and characteristics, and replacing it with a color code. The genetic algorithm represents the characteristics of game entities as genes on chromosomes and, through two fitness functions, evaluates the player's performance and choices to generate personalized content that meets their specific needs. In the case of the SGH for managing T1DM and childhood obesity, the algorithm was modified to utilize data from physical activity sensors and adapt game recommendations and missions according to the child's daily habits. This approach allows for dynamic adjustment of difficulty and educational content, optimizing user engagement and intervention effectiveness, as demonstrated by the results of experimental evaluations.

The third axis of the proposed framework focuses on the testing of dynamic SGH content and the PCG techniques with the help of deep reinforcement learning agents. Initially, content evaluation is examined through player modeling based on biofeedback sensors and subjective player reports. As the experimental process often requires human testing of the SGHs, a machine learning model is designed to enhance the value of live experiments with real players. The model is trained on screen recording data from sessions along with player annotations for indicators of interest to predict indicators in future sessions. The generated data and sensor data enable more efficient estimation and correlation of indicators with the target indicators of the SGH. The application of this testing in the SGH for dietary behaviors in order to model player engagement highlighted the need for further study of the architecture and training process of the machine learning model. In the SGH for stress management, the modeling of the stress indicator was possible, as demonstrated by the analysis of measurements recorded by photoplethysmography sensors, with high correlation values between the measured and estimated values of stress.

With regard to the applied PCG techniques, their testing traditionally requires extensive sessions with humans, a process that is time-consuming, costly, and often prone to subjective errors. The proposed framework automates this process, consisting of three main spaces: the SGH space, which encapsulates the logic of the game, the Interaction Interpreter space, which translates the game mechanisms into a form suitable for agents, and the Deep Reinforcement Learning space, where agent training and evaluation take place. The Game Description Language and the Gymnasium environment are used to standardize the interaction between the SGH and the agents, while the Stable Baselines 3 library is used to implement the deep reinforcement learning algorithms. The evaluation methodology includes two validation procedures: exhaustive validation, where trained agents encounter a wide range of procedurally generated content, and scenario-based validation, where they encounter realistic versions of the content controlled exclusively by the PCG. The results of applying the framework to the SGH for OSA confirm the improved performance of genetic algorithm versions by 5% compared to random content generation, with agents reaching the 50% win-rate threshold earlier during training. In addition, a direct correlation was observed between changes in the dominant genes of the genetic algorithm and changes in the success rate of the agents, confirming the effectiveness of the adaptation mechanism. In addition to automated testing, experimental procedures are conducted with real users, demonstrating the positive effect of the technique on user experience, with a significant increase in the feeling of competence (2.45±0.71 vs. 2.13±0.84, p=0.033) and a reduction in negative experience (0.02±0.08 vs. 0.11±0.17, p=0.020) in the SGH for OSA, as well as in clinical outcomes, with a statistically significant reduction in BMI z-score (mean reduction -0.21±0.26, p<0.001) in the SGH for T1DM and childhood obesity management. The findings of the thesis indicate that the application of the framework in the development of SGHs can lead to more effective interventions, both in terms of user experience and clinical outcomes, contributing to the promotion of self-management of chronic diseases and the wider adoption of SGHs as complementary tools in the arsenal of modern health interventions.

In summary, the framework presented in this thesis offers a methodological approach to the management, generalization, and testing of content in SGHs, addressing key challenges in the development of digital health interventions. The three axes of the framework aim to reduce the time and cost of development, while supporting the personalization of content and the enhancement of the intervention. The application of the framework in different health fields indicates its potential for generalization, while the results of the experiments provide encouraging evidence of its contribution to improving user experience and clinical outcomes, supporting further exploration and development of similar approaches in the field of digital health interventions.

Supervisor: Konstantina S. Nikita

PhD Student: Eleftherios Kalafatis