Traditional screening practices for anxiety and depression pose an impediment to monitoring and treating these conditions effectively. However, recent advances in NLP and speech modelling allow textual, acoustic, and hand-crafted language-based features to jointly form the basis of future mental health screening and condition detection. Speech is a rich and readily available source of insight into an individual's cognitive state and by leveraging different aspects of speech, we can develop new digital biomarkers for depression and anxiety. To this end, we propose a multi-modal system for the screening of depression and anxiety from self-administered speech tasks. The proposed model integrates deep-learned features from audio and text, as well as hand-crafted features that are informed by clinically-validated domain knowledge. We find that augmenting hand-crafted features with deep-learned features improves our overall classification F1 score comparing to a baseline of hand-crafted features alone from 0.58 to 0.63 for depression and from 0.54 to 0.57 for anxiety. The findings of our work suggest that speech-based biomarkers for depression and anxiety hold significant promise in the future of digital health.

Despite the impact of psychiatric disorders on clinical health, early-stage diagnosis remains a challenge. Machine learning studies have shown that classifiers tend to be overly narrow in the diagnosis prediction task. The overlap between conditions leads to high heterogeneity among participants that is not adequately captured by classification models. To address this issue, normative approaches have surged as an alternative method. By using a generative model to learn the distribution of healthy brain data patterns, we can identify the presence of pathologies as deviations or outliers from the distribution learned by the model. In particular, deep generative models showed great results as normative models to identify neurological lesions in the brain. However, unlike most neurological lesions, psychiatric disorders present subtle changes widespread in several brain regions, making these alterations challenging to identify. In this work, we evaluate the performance of transformer-based normative models to detect subtle brain changes expressed in adolescents and young adults. We trained our model on 3D MRI scans of neurotypical individuals (N=1,765). Then, we obtained the likelihood of neurotypical controls and psychiatric patients with early-stage schizophrenia from an independent dataset (N=93) from the Human Connectome Project. Using the predicted likelihood of the scans as a proxy for a normative score, we obtained an AUROC of 0.82 when assessing the difference between controls and individuals with early-stage schizophrenia. Our approach surpassed recent normative methods based on brain age and Gaussian Process, showing the promising use of deep generative models to help in individualised analyses.

Emotions are experienced and expressed through various response systems. Coherence between emotional experience and emotional expression is considered highly important to clients' well being. To date, emotional coherence has been studied at a single time point using lab-based tasks with relatively small datasets. No study has examined emotional coherence between the subjective experience of emotions and utterance-level emotions over therapy sessions or whether this coherence is associated with clients' well being. Natural language Processing (NLP) approaches have been applied to identify emotions during psychotherapy dialogue, which can be implemented to study emotional processes on a larger scale and with specificity. However, these methods have yet to be used to study coherence between emotional experience and emotional expression over the course of therapy and whether it relates to clients' well-being.This work presents an end-to-end approach where we use emotion predictions from our transformer based emotion recognition model to study emotional coherence and its diagnostic potential in psychotherapy research. We first employ our transformer based approach on a Hebrew psychotherapy dataset to automatically label clients' emotions at the utterance level in psychotherapy dialogues. We subsequently investigate the emotional coherence between clients' self-reported emotional states and our model-based emotion predictions. We also examine the association between emotional coherence and clients' well being.The findings indicate a significant correlation between clients' self-reported emotions and positive and negative emotions expressed verbally during psychotherapy sessions. Coherence in positive emotions was also highly correlated with clients well-being. These results illustrate how NLP can be applied to identify important emotional processes in psychotherapy to improve diagnosis and treatment for clients who suffer from mental-health problems.

Social media is a large and growing feature of teen life across the world. While some research suggests that social media are partlyto blame for growing rates of mental illness among teens, social media can also play a positive role in promoting teen mental healthby giving teens new ways to socialize and feel part of a community. In this work, we propose a framework for developing system that can further enhance the upsides of social media use: a computational model that uses social media data to predict depression, as part of a detection-and-intervention loop that engages the user in positive conversations when dynamic indicators of depression present themselves in their social media activity. Our framework uses three pillars of multimodal Content, Behavioral, and Contextual data drawn from users’ social media feeds in order to provide timely detection and intervention services via a chatbot. This multimodal architecture allows us to envision chat features that are precise and responsive to the behavior that triggers the detection. We present a review of the state of the art in depression detectionsystems, and then proceed to explain our system, which builds upon successes in deep learning-based detection systems, as well asplacing these tools in the new dynamic setting of online depression detection that enables our chatbot to initiate therapeutic interactions with social media users.

Word embeddings are extensively used in various NLP problems as a state-of-the-art semantic feature vector representation. Despite their success on various tasks and domains, they might exhibit an undesired bias for stereotypical categories due to statistical and societal biases that exist in the dataset they are trained on. In this study, we analyze the gender bias in four different pre-trained word embeddings specifically for the depression category in the mental disorder domain. We use contextual and non-contextual embeddings that are trained on domain-independent as well as clinical domain-specific data. We observe that embeddings carry bias for depression towards different gender groups depending on the type of embeddings. Moreover, we demonstrate that these undesired correlations are transferred to the downstream task for depression phenotype recognition. We find that data augmentation by simply swapping gender words mitigates the bias significantly in the downstream task.

Mental health and behavioural problems are the primary drivers of disability worldwide. Further escalated by the COVID-19 pandemic, millions across the globe are breaking traditional stigma by seeking professional support for their mental health. However, this increased demand for mental healthcare needs to be met by a limited number of services and professionals. We conducted qualitative interviews with mental health practitioners to understand the landscape of opportunities and challenges for AI-enabled mental healthcare in 2022, focussing on triage and decision support. Our findings suggest important opportunities for AI to accommodate the growing demand for mental healthcare, support clinicians’ workload, and improve data management. However, there were also major challenges identified regarding practitioner trust in AI solutions and their incorporation into the care pathway. Our findings indicate a need for coordinated training and education for mental health professionals to improve trust in AI solutions and correspondingly facilitate wider adoption of this promising technology. Moreover, a re-positioning of AI solutions as decision support tools rather than absolute decision tools might lead to improved acceptance and adoption within the clinical community. Finally, our results highlight the importance of understanding the end-user's perspective (in this case, mental health practitioners) and including them in the process of developing AI solutions in order to achieve optimal real-world impact.

Context: Mental healthcare systems are facing an ever-growing demand for appropriate assessment and intervention. Unfortunately, services are often centralized, overloaded, and inaccessible, resulting in greater institutional and social inequities. Therefore, there is an urgent need to establish easy-to-implement methods for early diagnosis and personalized follow-up. In recent years, serious games have started to offer such a clinical tool at scale.Problem: There are critical challenges to the development of secure and inclusive serious games for clinical research. First, the quality of the data and features analyzed must be well defined early in the research process in order to draw meaningful conclusions. Second, algorithms must be aligned with the purpose of the research while not perpetuating bias. Finally, the technologies used must be widely accessible and sufficiently engaging for users.Focus of the paper: To tackle these challenges, we designed a participatory project that combines three innovative technologies: Mixed Reality, Serious Gaming, and Machine Learning. We analyze preliminary data with a focus on the identification of the players and the measurement of classical biases such as sex and environment of data collection. Method: We co-developed with patients and their families, as well as clinicians, a serious game in mixed reality specifically designed for evaluation and therapeutic intervention in autism. Preliminary data were collected from neurotypical individuals with a mixed reality headset. Relevant behavioral features were extracted and used to train several classification algorithms for player identification. Results: We were able to classify players above chance with slightly higher accuracy of neural networks. Interestingly, the accuracy was significantly higher when players were separated by sex. Furthermore, the uncontrolled condition showed better levels of accuracy than the controlled condition. This could mean that the data are richer when the player interacts freely with the game. Our proof of concept cannot exclude the possibility that this last result is linked to the experimental setup. Future development will clarify this point with a larger sample size and the use of deep learning algorithms. Implications: We show that serious games in mixed reality can be a valuable tool to collect clinical data. Our preliminary results highlight important biases to consider for future studies, especially for the sex and context of data collection. Next, we will evaluate the usability, accessibility, and tolerability of the device and the game in autistic children. In addition, we will evaluate the psychometric properties of the serious game, especially for patient stratification. This project aims to develop a platform for the diagnosis and therapy of autism, which can eventually be easily extended to other conditions and settings such as the evaluation of depression or stroke rehabilitation. Such a tool can offer novel possibilities for the study, evaluation, and treatment of mental conditions at scale, and thus ease the burden on healthcare systems.

Avoidant/Restrictive Food Intake Disorder (ARFID), a new diagnosis in the DSM-5, is an eating disorder that can emerge in early childhood, threatens optimal physical growth and social-emotional development, and has been reported to persist, for some, well into adolescence or adulthood. Food selectivity more broadly has been reported to be more elevated in families of lower income, while the accessibility and affordability of treatment for mental health patients in the underrepresented group are limited. Therefore, it is crucial to develop accessible, affordable, and effective therapies. We designed a unique clinical study that can be implemented at home, which provides patients with a framework to work towards overcoming the challenges associated with ARFID. During the intervention, participants are filmed and relevant facial information is collected, automatically analyzed with machine learning and computer vision, and delivered to medical experts to enhance the knowledge they use for clinical judgment. We automatically extract affect-related features right after the participants taste or smell a food they labeled as moderately challenging. We observed that facial action units activation provides interesting patterns helpful in understanding the patient’s experience throughout the food exposure treatment. This rich information enables quantification of the effectiveness of the currently investigated treatments and differentiation of patient-specific responses to them, potentially leading to scalable personalized medicine for ARFID.

In this work, we explore the effectiveness of two different acoustic feature groups - conventional hand-curated and deep representation features, for predicting the severity of depression from speech. We measure the relevance of possible contributing factors to the models' performance, including gender of the individual, severity of the disorder, content and length of speech. Our findings suggest that models trained on conventional acoustic features perform equally well or better than the ones trained on deep representation features at significantly lower computational cost, irrespective of other factors, e.g. content and length of speech, gender of the speaker and severity of the disorder. This makes such models a better fit for deployment where availability of computational resources is restricted, such as real time depression monitoring applications in smart devices.

Machine learning models often request personal information from users to assign more accurate predictions across a heterogeneous population. Personalized models are not built to support \emph{informed consent}: users cannot "opt-out" of providing personal data, nor understand the effects of doing so. In this work, we introduce a family of personalized prediction models called \emph{participatory systems} that support informed consent. Participatory systems are interactive prediction models that let users opt into reporting additional personal data at prediction time, and inform them about how their data will improve their predictions. We present a model-agnostic approach for supervised learning tasks where personal data is encoded as "group" attributes (e.g., sex, age group, HIV status). Given a pool of user-specified models, our approach can create a variety of participatory systems that differ in their training requirements and opportunities for informed consent. We conduct a comprehensive empirical study of participatory systems in clinical prediction tasks and compare them to common approaches for personalization. Our results show that our approach can produce participatory systems that exhibit large improvements in privacy, fairness, and performance at the population and group levels.

There has been considerable research in the field of automated mental health analysis. Studies based on patient-therapist interviews usually treat the dyadic discourse as a sequence of sentences, thus ignoring individual sentence types (question or answer). To avoid this situation, we design a multi-view architecture that retains the symmetric discourse structure by dividing the transcripts into patient and therapist views. Experiments on the DAIC-WOZ dataset for depression level rating show performance improvements over baselines and state-of-the-art models.

In this paper, we present a framework that explains the scores of sketches by learning rules used in developmental tests. To achieve this, we propose a deep neural network model that considers a target and the corresponding sketch images as inputs. The proposed method is divided into plain and residual models according to the presence of residual connections to compare their performance. In addition, each model includes the subtraction and concatenation approaches to fuse two feature maps. To verify the performance of the proposed method, we conduct experiments over all settings that combine the proposed models with the fusion method. The results show that the proposed framework can be used for visual-motor integration analysis by determining scores and providing explanations.

Traditional screening practices for anxiety and depression pose an impediment to monitoring and treating these conditions effectively. However, recent advances in NLP and speech modelling allow textual, acoustic, and hand-crafted language-based features to jointly form the basis of future mental health screening and condition detection. Speech is a rich and readily available source of insight into an individual's cognitive state and by leveraging different aspects of speech, we can develop new digital biomarkers for depression and anxiety. To this end, we propose a multi-modal system for the screening of depression and anxiety from self-administered speech tasks. The proposed model integrates deep-learned features from audio and text, as well as hand-crafted features that are informed by clinically-validated domain knowledge. We find that augmenting hand-crafted features with deep-learned features improves our overall classification F1 score comparing to a baseline of hand-crafted features alone from 0.58 to 0.63 for depression and from 0.54 to 0.57 for anxiety. The findings of our work suggest that speech-based biomarkers for depression and anxiety hold significant promise in the future of digital health.

Emotions are experienced and expressed through various response systems. Coherence between emotional experience and emotional expression is considered highly important to clients' well being. To date, emotional coherence has been studied at a single time point using lab-based tasks with relatively small datasets. No study has examined emotional coherence between the subjective experience of emotions and utterance-level emotions over therapy sessions or whether this coherence is associated with clients' well being. Natural language Processing (NLP) approaches have been applied to identify emotions during psychotherapy dialogue, which can be implemented to study emotional processes on a larger scale and with specificity. However, these methods have yet to be used to study coherence between emotional experience and emotional expression over the course of therapy and whether it relates to clients' well-being.This work presents an end-to-end approach where we use emotion predictions from our transformer based emotion recognition model to study emotional coherence and its diagnostic potential in psychotherapy research. We first employ our transformer based approach on a Hebrew psychotherapy dataset to automatically label clients' emotions at the utterance level in psychotherapy dialogues. We subsequently investigate the emotional coherence between clients' self-reported emotional states and our model-based emotion predictions. We also examine the association between emotional coherence and clients' well being.The findings indicate a significant correlation between clients' self-reported emotions and positive and negative emotions expressed verbally during psychotherapy sessions. Coherence in positive emotions was also highly correlated with clients well-being. These results illustrate how NLP can be applied to identify important emotional processes in psychotherapy to improve diagnosis and treatment for clients who suffer from mental-health problems.

According to the Global Burden of Disease list provided by the WHO, mental disorders are among the most debilitating disorders. To improve diagnosis and therapy effectiveness, in recent years, researchers tried to identify individual biomarkers. Gathering neurobiological data however, is costly and time-consuming. Another potential source of information, which is already part of the clinical routine, are therapist-patient dialogues. While there are some pioneering works investigating the role of language as predictors for various therapeutic parameters, for example patient-therapist alliance, there are no large-scale studies. A major obstacle to conduct these studies is the availability of sizeable datasets, which are needed to train machine learning models. While these conversations are part of the daily routine of clinicians, gathering them is usually hindered by various ethical (purpose of data usage), legal (data privacy) and technical (data formatting) limitations. Some of which are particular to the domain of therapy dialogues, like the increased difficulty in anonymisation, or the transcription of the recordings. In this paper, we elaborate on the challenges we faced in starting our collection of therapist-patient dialogues in a psychiatry clinic under the General Data Privacy Regulation of the European Union with the goal to use the data for NLP research. We give an overview over each step in our procedure and point out potential pitfalls to motivate further research in this field.

Despite the impact of psychiatric disorders on clinical health, early-stage diagnosis remains a challenge. Machine learning studies have shown that classifiers tend to be overly narrow in the diagnosis prediction task. The overlap between conditions leads to high heterogeneity among participants that is not adequately captured by classification models. To address this issue, normative approaches have surged as an alternative method. By using a generative model to learn the distribution of healthy brain data patterns, we can identify the presence of pathologies as deviations or outliers from the distribution learned by the model. In particular, deep generative models showed great results as normative models to identify neurological lesions in the brain. However, unlike most neurological lesions, psychiatric disorders present subtle changes widespread in several brain regions, making these alterations challenging to identify. In this work, we evaluate the performance of transformer-based normative models to detect subtle brain changes expressed in adolescents and young adults. We trained our model on 3D MRI scans of neurotypical individuals (N=1,765). Then, we obtained the likelihood of neurotypical controls and psychiatric patients with early-stage schizophrenia from an independent dataset (N=93) from the Human Connectome Project. Using the predicted likelihood of the scans as a proxy for a normative score, we obtained an AUROC of 0.82 when assessing the difference between controls and individuals with early-stage schizophrenia. Our approach surpassed recent normative methods based on brain age and Gaussian Process, showing the promising use of deep generative models to help in individualised analyses.

Schizophrenia spectrum disorders (SSZ) affect more than 24 million individuals worldwide. They present an acute onset of psychotic symptoms such as delusions, hallucinations, perceptual disturbances, and severe disruption of ordinary behavior which affect the wellbeing of individuals. Despite recent advanced in risk prediction models, there remains important gaps in the literature, particularly a lack of evaluation with large samples and external datasets, as well as concerns regard potential bias and discrimination. Furthermore, the current state-of-the art risk models are based on electronic health records, electroencephalograms, and genetic data, which are acquired in medical centres using expensive equipment, hence limiting widespread access to such tools by the general population. Hence, novel fair models to identify individuals at high risk and modifiable risk factors are essential to improve risk prediction of SSZ.To tackle these limitations, we developed and validated a novel, accessible and fair ML model for risk prediction of SSZ. From UK Biobank, a large longitudinal cohort, 591 participants who were diagnosed with schizophrenia, schizotypal and delusional disorders after the baseline assessment visit, were identified and included in our study. An equal number of healthy participants were selected as the control group by matching age and sex using propensity scores. This resulted in a total of 1182 participants being selected for our study; 1064 participants from 18 of the 22 UK Biobank assessment centers were used in nested cross-validation, and 306 participants from the remaining four centers were selected for external validation. We considered data from the participants’ baseline visit and selected 198 factors related to life course exposures, blood biochemistry and haematology. Subsequently, we performed data imputation to account for missing patient data. We evaluated different machine learning models to identify individuals at risk of schizophrenia spectrum disorders after the baseline visit: Logistic Regression, Support Vector Machines, Random Forest, AdaBoost and XGBoost. We assessed models’ performance in terms of AUC, F1-Score, precision, and sensitivity. Moreover, we evaluated the fairness of the best performing models by means of statistical parity difference and disparate impact ratio to identify and mitigate potential biases related to ethnicity, sex, birth, education and material deprivation. We interpreted the results by estimating feature importance using the SHapley Additive exPlanations (SHAP) values.Our results demonstrate that machine learning models based on accessible exposome variables such as Townsend deprivation and diet, can reliably identify individuals at risk of schizophrenia, schizotypal and delusional disorders. Haematological data slightly improve the results in terms of accuracy. For the task at hand, XGBoost outperforms other models with the best fair model achieving an AUC of 0.822 and 0.796 in internal and external validation cohorts, respectively. These preliminary results show promise for further investigation of accessible and fair ML models in mental health that will benefit the general population across various ethnic, sex, age and socio-economics groups.

Word embeddings are extensively used in various NLP problems as a state-of-the-art semantic feature vector representation. Despite their success on various tasks and domains, they might exhibit an undesired bias for stereotypical categories due to statistical and societal biases that exist in the dataset they are trained on. In this study, we analyze the gender bias in four different pre-trained word embeddings specifically for the depression category in the mental disorder domain. We use contextual and non-contextual embeddings that are trained on domain-independent as well as clinical domain-specific data. We observe that embeddings carry bias for depression towards different gender groups depending on the type of embeddings. Moreover, we demonstrate that these undesired correlations are transferred to the downstream task for depression phenotype recognition. We find that data augmentation by simply swapping gender words mitigates the bias significantly in the downstream task.

Machine learning models often request personal information from users to assign more accurate predictions across a heterogeneous population. Personalized models are not built to support \emph{informed consent}: users cannot "opt-out" of providing personal data, nor understand the effects of doing so. In this work, we introduce a family of personalized prediction models called \emph{participatory systems} that support informed consent. Participatory systems are interactive prediction models that let users opt into reporting additional personal data at prediction time, and inform them about how their data will improve their predictions. We present a model-agnostic approach for supervised learning tasks where personal data is encoded as "group" attributes (e.g., sex, age group, HIV status). Given a pool of user-specified models, our approach can create a variety of participatory systems that differ in their training requirements and opportunities for informed consent. We conduct a comprehensive empirical study of participatory systems in clinical prediction tasks and compare them to common approaches for personalization. Our results show that our approach can produce participatory systems that exhibit large improvements in privacy, fairness, and performance at the population and group levels.