Deep Learning for Drug Pair Scoring

In this talk I discuss ChemicalX, a PyTorch-based deep learning library designed for providing a range of state of the art models to solve the drug pair scoring task. The primary objective of the library is to make deep drug pair scoring models accessible to machine learning researchers and practitioners in a streamlined framework.The design of ChemicalX reuses existing high level model training utilities, geometric deep learning, and deep chemistry layers from the PyTorch ecosystem. Our system provides neural network layers, custom pair scoring architectures, data loaders, and batch iterators for end users. These features are showcased with example code snippets and case studies to highlight the characteristics of ChemicalX. A range of experiments on real world drug-drug interaction, polypharmacy side effect, and combination synergy prediction tasks demonstrate that the models available in ChemicalX are effective at solving the pair scoring task. Finally, it is shown that ChemicalX could be used to train and score machine learning models on large drug pair datasets with hundreds of thousands of compounds on commodity hardware.

Modeling and characterizing multiple factors is perhaps the most important step in achieving excess returns over market benchmarks. Both academia and industry are striving to find new factors that have good explanatory power for future stock returns and good stability of their predictive power. In practice, factor investing is still largely based on linear multi-factor models, although many deep learning methods show promising results compared to traditional methods in stock trend prediction and portfolio risk management. However, the existing non-linear methods have two drawbacks: 1) there is a lack of interpretation of the newly discovered factors, 2) the financial insights behind the mining process are unclear, making practitioners reluctant to apply the existing methods to factor investing. To address these two shortcomings, we develop a novel deep multi-factor model that adopts industry neutralization and market neutralization modules with clear financial insights, which help us easily build a dynamic and multi-relational stock graph in a hierarchical structure to learn the graph representation of stock relationships at different levels, e.g., industry level and universal level. Subsequently, graph attention modules are adopted to estimate a series of deep factors that maximize the cumulative factor returns. And a factor-attention module is developed to approximately compose the estimated deep factors from the input factors, as a way to interpret the deep factors explicitly. Extensive experiments on real-world stock market data demonstrate the effectiveness of our deep multi-factor model in the task of factor investing. We will release source codes upon publication to facilitate reproduction of our results.

Understanding stock market instability is a key question in financial managementas practitioners seek to forecast breakdowns in asset co-movements which exposeportfolios to rapid and devastating collapses in value. The structure of theseco-movements can be described as a graph where companies are represented bynodes and edges capture correlations between their price movements. Learning atimely indicator of co-movement breakdowns (manifested as modifications in thegraph structure) is central in understanding both financial stability and volatilityforecasting. We propose to use the edge-reconstruction accuracy of a graph auto-encoder (GAE) as an indicator for how spatially homogeneous connections betweenassets are, which, based on financial network literature, we use as a proxy to infermarket volatility. Our experiments on the S&P 500 over the 2015-2022 period showthat higher GAE reconstruction error values are correlated with higher volatility.We also show that out-of-sample autoregressive modeling of volatility is improvedby the addition of the proposed measure. Our paper contributes to the literatureof machine learning in finance particularly in the context of understanding stockmarket instability.

Valuing mineral assets is a challenging task that is highly dependent on the supply (geological) uncertainty surrounding resources and reserves, and the uncertainty of demand (commodity prices). In this work, a graph-based reasoning, modeling and solution approach is proposed to jointly address mineral asset valuation and mine plan scheduling and optimization under supply and demand uncertainty in the "mining complex" framework. Three graph-based solutions are proposed: (i) a neural branching policy that learns a block-sampling ore body representation, (ii) a guiding policy that learns to explore a heuristic selection tree, (iii) a hyper-heuristic that manages the value/supply chain optimization and dynamics modeled as a graph structure. Results on two large-scale industrial mining complexes show a reduction of up to three orders of magnitude in primal suboptimality, execution time, and number of iterations, and an increase of up to 40% in the mineral asset value.

Graph-structured data is ubiquitous throughout natural and social sciences, and Graph Neural Networks (GNNs) have recently been shown to be effective at solving prediction and inference problems on graph data. In this paper, we propose and demonstrate that GNNs can and should be applied to solve Combinatorial Optimization (CO) problems. Combinatorial Optimization (CO) concerns optimizing a function over a discrete solution space that is often intractably large. To learn to solve CO problems, we phrase specifying a candidate solution as a sequential decision making problem, where the return is related to how close the candidate solution is to optimality. We use a GNN to learn a policy to iteratively build increasingly promising candidate solutions. We present preliminary evidence that GNNs trained through Q-Learning can solve CO problems with performance approaching state-of-the-art heuristic-based solvers, using only a fraction of the parameters and training time.

Graph Neural Networks (GNNs) require a relatively large number of labeled nodes and a reliable/uncorrupted graph connectivity structure in order to obtain good performance on the semi-supervised node classification task. The performance of GNNs can degrade significantly as the number of labeled nodes decreases or the graph connectivity structure is corrupted by adversarial attacks or due to noises in data measurement/collection. Therefore, it is important to develop GNN models that are able to achieve good performance when there is limited supervision knowledge – a few labeled nodes and noisy graph structures. In this paper, we propose a novel Dual GNN learning framework to address this challenging task. The proposed framework has two GNN based node prediction modules. The primary module uses the input graph structure to induce regular node embeddings and predictions with a regular GNN baseline, while the auxiliary module constructs a new graph structure through fine-grained spectral clustering and learns new node embeddings and predictions. By integrating the two modules in a dual GNN learning framework, we perform joint learning in an end-to-end fashion. This general framework can be applied on many GNN baseline models. The experimental results validate that the proposed dual GNN framework can greatly outperform the GNN baseline methods when the labeled nodes are scarce, and the graph connectivity structure is noisy.

Graph AI to Enable Precision Medicine

Graph representation learning leverages knowledge, geometry, and structure to develop powerful machine learning methods. First, I will introduce Shepherd, a graph neural network for personalized diagnosis of patients with rare genetic diseases. Diagnostic delay is pervasive in patients with rare genetic conditions. It can lead to numerous problems, including redundant testing and unnecessary procedures, delays in obtaining disease-appropriate management and therapies, and even irreversible disease progression. Shepherd uses knowledge-guided geometric deep learning to gather information from different parts of a knowledge graph and logically connect a patient's clinical-genomic information to the region in the knowledge graph relevant to diagnosis. Evaluation of patients from the Undiagnosed Diseases Network shows that Shepherd accurately identifies causal disease genes, finds other patients with the same causal gene and disease, and provides interpretable characterizations of novel diseases. Second, I will describe applications of graph neural networks in drug discovery. These are available through Therapeutics Data Commons (https://tdcommons.ai), an initiative to access and evaluate AI capability across therapeutic modalities and stages of drug discovery. The Commons supports the development of machine learning methods, with a strong bent towards developing the foundations for which methods are most suitable for drug discovery and why.

The majority of biological functions are carried out by proteins. Proteins perform their roles only upon arrival to their target location in the cell, hence elucidating protein sub-cellular localization is essential for better understanding their function. The exponential growth in genomic information and the high cost of experimental validation of protein localization call for the development of predictive methods. We present a method that improves sub-cellular localization prediction for proteins based on their sequence by leveraging structure prediction and Graph Neural Networks. We demonstrate how Language Models, trained on protein sequences, and Graph Neural Networks, trained on protein's 3D structures, are both efficient approaches for this task. They both learn meaningful, yet different representations of proteins; hence, ensembling them outperforms the reigning state of the art method. Our architecture improves the localization prediction performance while being lighter and more cost-effective.

Neural networks are increasingly used in finance, and complex systems. One of the relevant characteristics of financial systems, and other complex systems, is the intricate dependency structure between variables. Such a dependency structure is a high-order network of relations and it is very important to be able to represent it within the models’ architecture. Traditional neural networks have not been designed to capture such complexity and its dynamics. We propose to investigate the design and testing of a novel deep learning neural network architecture, termed Homological Neural Network (HNN) with a higher-order graphical structure that can adjust dynamically to better model the multivariate dynamical complexity of data with practical applications in finance and more.

Battery life estimation is critical for optimizing battery performance and guaranteeing minimal degradation for better efficiency and reliability of battery-powered systems. The existing methods to predict the Remaining Useful Life(RUL) of Lithium-ion Batteries (LiBs) neglect the relational dependencies of the battery parameters to model the nonlinear degradation trajectories. We present the Battery GraphNets framework that jointly learns to incorporate a discrete dependency graph structure between battery parameters to capture the complex interactions and the graph-learning algorithm to model the intrinsic battery degradation for RUL prognosis. The proposed method outperforms several popular methods by a significant margin on publicly available battery datasets and achieves SOTA performance. We report the ablation studies to support the efficacy of our approach.

Abstract removed upon request by authors.

Cancers are shaped by somatic mutations, microenvironment, and patient background, each altering both gene expression and gene regulatory networks (GRNs) in complex ways, resulting in highly-variable cellular states and dynamics. Inferring GRNs from expression data can help characterize this regulation-driven heterogeneity, but network inference is intractable without many statistical samples, limiting GRNs to cluster-level analyses that ignore intra-cluster heterogeneity. We propose to move beyond cluster-based analyses by using \emph{contextualized} learning, a meta-learning paradigm, to generate sample-specific network models from sample contexts. We unify three network classes (correlation, Markov, Bayesian) and estimate sample-specific GRNs for 7000 tumours across 25 tumor types, with each network contextualized by copy number and driver mutation profiles, tumor microenvironment and patient demographics. Sample-specific networks provide a de-noised view of gene expression dynamics at sample-specific resolution, which reveal co-expression modules in correlation networks (CNs), clique structures and neighborhood selection in Markov Networks (MNs), and causal ordering and probability factorization in Bayesian Networks (BNs). Sample-specific networks enable GRN-based precision oncology, including brain tumor subtyping that improves survival prognosis.