Extensive modeling work on the BOUNCE study dataset was instrumental in identifying key predictors of well-being trajectories during BC treatments and recovery. Machine Learning (ML) models displayed sufficient predictive accuracy regarding the probability that a given patient will show a successful recovery (i.e., achieve a fair level of well-being and escape the relatively infrequent possibility of a significant decline in well-being at the end of the arduous period of BC treatments).
Yet, clinicians who are called in to support women likely to show poor illness adaptation are often challenged with the task of choosing therapeutic targets that can be directly and efficiently pursued in the context of relatively brief psychological interventions. In order to meet these clinical needs, the BOUNCE clinicians worked closely with the modeling team at FORTH to redesign the BOUNCE Decision Support Tool (DST) according to four principles: (i) flexibility toward future use in clinical settings, (ii) performance accuracy in predicting key aspects of patient well-being, (iii) robustness in formulating personalized risk profiles of potentially modifiable patient characteristics, and (iv) directly linking personalized needs assessment with concrete suggestions regarding psychological prevention strategies.
Flexibility means that clinicians can adapt the DST to their clinical needs and to the availability of data for a given patient. Along these lines, we included models capable of assessing the risk for six distinct well-being outcomes (illustrated in the following diagram). Two models address the need to identify patients at risk of overall poor mental health at a particular end-time. These models were deemed more appropriate in terms of classification performance regardless of the patient’s reported mental health at the time of diagnosis and at the beginning of cancer treatments. In other words, they can be applied to predict future mental health status regardless of how the patient felt initially.
Two alternative models address the need to identify patients at risk of declining mental health having displayed adequate classification performance in the subgroup of patients who reported good mental health at the time of diagnosis (and/or treatment onset).
The DST provides the option to set the overall patient quality of life (QoL) as an alternative clinical outcome
Additional clinical flexibility is afforded in terms of the available psychological and lifestyle measurements in mainstream clinical practice. Thus, it is possible to perform patient risk assessment under three alternative clinical scenarios in assessing risk for poor psychosocial recovery:
- Combining clinical and biological information with patient-reported psychological characteristics at the time of diagnosis and at 3 months later,
- Combining clinical and biological information with patient-reported psychological characteristics obtained at 6 months post-diagnosis, and
- Combining clinical and biological information with patient-reported psychological characteristics aggregated over the first 6 months post-diagnosis.
The performance principle means that the selected models were those that displayed adequate classification accuracy through extensive cross-validation schemes on the BOUNCE prospective clinical study. Personalized risk profiles can be obtained through a novel type of AI models, known as agnostic (or local explainability) analyses at the patient level1,2. This approach can pinpoint specific psychological or lifestyle characteristics, which appear to be underdeveloped in a given patient based on available and appropriately timed measurements. Model-agnostic analysis works by searching for predictor variables that contribute the most to the risk assessment for a given patient after statistically controlling for all other predictors in the model. An example of a personalized prediction of significant decline in overall mental health for a patient who at the time of diagnosis reported fair psychological status is given below
The 15 most highly ranked features selected by the Machine Learning model for this patient include 9 variables that appear to predominantly “facilitate” the adverse mental health outcome: relatively high negative affectivity, intense anxious preoccupation, and a sense of helplessness, combined with relatively low scores on Future Perspectives (that is, how worried patients are about their health in the future) and constitutional resilience. Conversely, the experience of only mild treatment side effects and negative affectivity, combined with high levels of a positive body image and mindfulness at the time of diagnosis, appear to exert a protective role for this patient by reducing the probability of an adverse mental health outcome.
Break-down profile of a patient who was correctly predicted by the ML model to display a substantial decline in overall mental health 12 months following diagnosis of breast cancer. Actual patient scores on each predictor variable are shown on the left-hand side
Having extracted potentially crucial predictor variables, the DST then compares the patient’s actual scores on these variables to the recorded scores of all patients in the BOUNCE prospective clinical study. Next, the DST identifies variables where the patient’s score is either very high (for “risk” variables, such as anxious preoccupation) or very low (for “protective” variables, such as coping strategies). Variables that meet these criteria are “flagged” as potential targets of prevention strategies by mental health professionals.
Based on the personalized risk and vulnerability profiles, the DST provides specific clinical recommendations each targeting a specific underdeveloped or deficient psychological or lifestyle characteristic of the patient. The platform integrates appropriate recommendations for a given patient into a single document in two versions:
- One (abbreviated) version addressed to clinicians who come in direct contact with the patient but are not trained in administering systematic psychological support (such as physicians, nurses, and social workers).
- An extended version is also available for use by mental health professionals who have some training in psychological interventions.
In sum, the DST user will have several options based on their clinical needs—in terms of both prediction endpoints and capabilities to engage diverse prevention strategies—and also according to the timing of available psychological and lifestyle data. These features are expected to facilitate the applicability of the DS platform for a wider variety of clinical scenarios and settings.
References
1Biecek, Przemysław. “DALEX: explainers for complex predictive models in R.” The Journal of Machine Learning Research 19.1 (2018): 3245-3249.
2Baniecki, Hubert, et al. “dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python.” arXiv preprint arXiv:2012.14406 (2020).