Panelists: Rocio Mercado - Incoming Assistant Professor Chalmers University Katherine Sytwu - Postdoc Lawrence Berkeley National Lab Felix Streith-Kalthoff - Postdoc University of Toronto Tian Xie - Researcher Microsoft AI4Science

The Open MatSci ML Toolkit is a flexible, self-contained and scalable Python-based framework to apply deep learning models and methods on scientific data with a specific focus on materials science and the OpenCatalyst Dataset. The primary components of our toolkit include: 1.Scalable computation of experiments leveraging PyTorch Lightning across different computation capabilities (laptop, server, cluster) and hardware platforms (CPU, GPU, XPU) without sacrificing performance in the compute and modeling; 2. Support for DGL for rapid graph neural network development. By sharing this toolkit with the research community via open-source release, we aim to: 1. Ease of use for new machine learning researchers and practitioners that want get started on interacting with the OpenCatalyst dataset which currently makes up the largest computational materials science dataset; 2. Enable the scientific community to apply advanced machine learning tools to high-impact scientific challenges, such as modeling of materials behavior for climate change applications.

Discovering a rare material within a vast search space exhibits a Needle-in-a-Haystack challenge. This challenge of finding a rare material, i.e., the "needle", inside a vast search space, i.e., the "haystack", arises when there is an extreme imbalance of optimum conditions relative to the size of the search space. For example, only 0.82% out of 146k total materials in the open-access Materials Project database have a negative Poisson's ratio, a rare material property. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence to a global optimum or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI, that builds on conventional Bayesian optimization principles to quickly and efficiently optimize Needle-in-a-Haystack problems in both less time and fewer experiments by addressing the common convergence and pigeonholing issues. ZoMBI actively extracts knowledge from the previously best-performing evaluated experiments to iteratively zoom in the sampling search bounds towards the global optimum "needle" and then prunes the memory of low-performing historical experiments to accelerate compute times by reducing the algorithm time complexity from $O(n^3)$ to $O(1)$, as the number of experiments sampled increases. Additionally, ZoMBI implements two custom acquisition functions that use active learning to further guide the sampling of new experiments towards the global optimum. We validate the algorithm's performance on two real-world 5-dimensional Needle-in-a-Haystack material property optimization datasets: discovery of auxetic Poisson's ratio materials and discovery of high thermoelectric figure of merit materials. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering materials in under 100 experiments that are up to 3x more highly optimized than those discovered by current state-of-the-art algorithms.

Architected materials that consist of multiple subelements arranged in particular orders can demonstrate a much broader range of properties than their constituent materials. However, the rational design of these materials generally relies on experts’ prior knowledge and requires painstaking effort. Here, we present a data efficient method for the multiproperty optimization of 3D-printed architected materials utilizing a machine learning (ML) cycle consisting of the finite element method (FEM) and 3D neural networks. Specifically, we applied our method to orthopedic implant design. Compared to expert designs, our experience-free method designed microscale heterogeneous architectures with a biocompatible elastic modulus and higher strength. Furthermore, inspired by the knowledge learned by the neural networks, we developed machine-human synergy, adapting the ML-designed architecture to fix a macroscale, irregularly shaped animal bone defect. Such adaptation exhibits 20 % higher experimental load-bearing capacity than the expert design. Thus, our method opens a new paradigm for the fast and intelligent design of architected materials with tailored mechanical, physical, and chemical properties.

Structure-based virtual screening (SBVS) of ultra-large chemical libraries has led to the discovery of novel inhibitors for challenging protein targets. However, screening campaigns of these magnitudes are expensive and thus impractical to employ in standard practice. As the broad goal of most SBVS workflows is the identification of the most potent compounds in the library, the task can be viewed as an optimization problem. Previous work has demonstrated the ability for Bayesian optimization to improve sample efficiency in SBVS using the MolPAL software. In this tutorial, we provide a broad algorithmic overview of the MolPAL software and a guide for its utilization in a prospective virtual screening task.

Abstract: The ability to quickly iterate through design cycles relies on our ability to synthesize molecules and materials found in virtual libraries or proposed by generative models. In this talk, we will focus on a primary consideration of molecular design workflows: the chemical space that comprises the search space for a molecular screening/optimization campaign. That is, the manner in which the search is constrained to a finite library of molecules or, following an increasingly popular trend, the manner in which the search navigates a virtually infinite space of molecules. We will talk about how models to predict chemical reactivity inform our ability to define and navigate these spaces. Further, we will discuss the challenge of sample efficiency, as most algorithms for molecular design operate in a regime that is severely misaligned with the reality of experimental work in terms of the number of candidates that can be evaluated within a practical budget. Bio: Connor W. Coley is an Assistant Professor at MIT in the Department of Chemical Engineering and the Department of Electrical Engineering and Computer Science. He received his B.S. and Ph.D. in Chemical Engineering from Caltech and MIT, respectively, and did his postdoctoral training at the Broad Institute. His research group at MIT develops new methods at the intersection of data science, chemistry, and laboratory automation to streamline discovery in the chemical sciences with an emphasis on therapeutic discovery. Key research areas in the group include the design of new neural models for representation learning on molecules, data-driven synthesis planning, in silico strategies for predicting the outcomes of organic reactions, model-guided Bayesian optimization, and de novo molecular generation. Connor is a recipient of C&EN’s “Talented Twelve” award, Forbes Magazine’s “30 Under 30” for Healthcare, the NSF CAREER award, and the Bayer Early Excellence in Science Award. Outside of MIT, Connor serves as an advisor to both early- and late-stage companies including Entos, Revela, Galixir, Kebotix, Anagenex, and Dow.

Synthesizing new inorganic functional materials is a practical goal of materials science. While the advances in computational techniques accelerated the virtual design, the actual synthesis of predicted candidate materials still remain as an expensive and slow process. While a few initial studies attempted to predict the synthesis routes for inorganic crystals, the existing models do not yield the uncertainty of the predictions and could produce thermodynamically unrealistic precursor chemicals. Here, we propose an element-wise graph neural network to predict the inorganic synthesis recipes. The trained model outperforms the popularity-based statistical baseline model for top-k exact match accuracy test, showing the validity of our approach for inorganic solid-state synthesis. We further validate our model by the publication-year-split test, where the model trained based on the materials data until the year 2016 is shown to successfully predict the synthetic precursors for the materials synthesized after 2016. The high correlation between the classification score and prediction accuracy suggests that the prediction score can be interpreted as a measure of uncertainty.

Rapid and in-depth exploration of the chemical space of high molecular weight synthetic polypeptides via the ring-opening polymerization (ROP) of N-carboxyanhydride (NCA) is a viable approach towards protein mimics and functional biomaterials. Here, we develop an efficient chemistry for the high throughput diversification of polypeptides based on a click-like reaction between selenolate and various electrophiles in aqueous solutions. With the assistance of automation and machine learning, iterative exploration of the random heteropolypeptides (RHPs) library efficiently and effectively identifies hit materials from a model system of which we have little prior knowledge. This automated and high-throughput platform provides a useful interface between wet and dry experiment, which would accelerate the discovery of new polypeptide materials for unmet challenges such as de novo design of artificial enzyme, biomacromolecule delivery, and understanding of intrinsically disordered proteins.

Retrosynthesis is one of the core tasks in the organic molecule design cycle, yet it is still a computational challenge to produce suitable sets of precursors for a desired product. Commonly used template-based approaches reduce the problem to a multi-class classification task for single steps. However, reactions in available datasets are noisy and incomplete, making usual training methods problematic. In this work, considering that multiple disconnections are possible for a product, we propose training models using differential top-k losses. We show that using these loss functions yields improvements in every top-N metric, with little overhead relative to cross-entropy. The use of more powerful models, more diverse and complete datasets, and other methodologies, is expected to yield significant improvements on this task when combined with the training approach presented here.

Solution-based coating methods offer a low-cost method for depositing coatings at scale. It is difficult, however, to obtain high quality coatings using these methods due to the complex physical phenomena at play. Here, we show how a self-driving laboratory can optimize a scalable spray-coating process applicable to manufacturing diverse clean-energy technologies. We demonstrate this system by optimizing a spray-combustion process for depositing conductive palladium films. This optimization yielded films with conductivities of 4.08 MS/m, doubling the state-of-the art film conductivity possible with this process and rivaling Pd film conductivities obtained using vacuum-based sputtering processes (2 to 6 MS/m). The rich data gathered by the self-driving laboratory also provides mechanistic insights into the coating process. The champion coating conditions were scaled up to an 8× larger area using the same spray-coating apparatus with no further optimization and no reduction in coating quality. This work shows how self-driving laboratories can optimize spray-coating for depositing coatings at scale.

Abstract: Deep learning schemes have already impacted areas such as cognitive game theory (e.g., computer chess and the game of Go), pattern (e.g., facial or fingerprint) recognition, event forecasting, and bioinformatics. They are beginning to make major inroads within physics, chemistry and materials sciences and hold considerable promise for accelerating the discovery of new theories and materials. In this talk, I will introduce deep convolutional neural networks and how they can be applied to the computer vision problems in transmission electron microscopy and tomographic imaging. Bio: Huolin Xin is a full professor at UC Irvine. He graduated from the Physics Department of Cornell University in 2011 and joined the University of California, Irvine in 2018. Prior to becoming a professor at UCI, he worked at Brookhaven National Laboratory as a scientific staff member and a principal investigator from 2013 to 2018. His research has resulted in more than 280 peer-reviewed publications (h-index 73 and citations 24,600). He received the MRS Oustanding Early Career Investigator Award, MSA Burton Medal, DOE Early Career Award, and the UCI Distinguished Early-Career Faculty for Research in 2020. He was the Chair of the largest international electron microscopy conference, Microscopy and Microanalysis, in 2020. His work on battery materials has been selected as the 2020, 2019 and 2014's Top-10 Scientific Achievements by Brookhaven Lab.

Materials discovery and design require characterizing material structure at the nanometer and sub-nanometer scale. Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM) resolves the crystal structure of materials, but many 4D-STEM data analysis pipelines are not suited for identification of anomalous and unexpected structures. This work introduces improvements to the iterative Non-Negative Matrix Factorization (NMF) method by implementing consensus clustering for ensemble learning. We evaluate the performance of models during parameter tuning and find that consensus clustering improves performance in all cases and is able to recover specific grains missed by the best performing model in the ensemble. The methods introduced in this work can be applied broadly to materials characterization datasets to aid in the design of new materials.

The evaluation of synthetic micro-structure images is an emerging problem as machine learning and materials science research have evolved together. Typical state of the art methods in evaluating synthetic images from generative models have relied on the Frechet Inception Distance. However, this and other similar methods, are limited in the materials domain due to the unique features that characterize physically accurate micro-structures and limited dataset sizes. In this study we evaluate a variety of methods on scanning electron microscope (SEM) images of graphene-reinforced polyurethane foams. The primary objective of this paper is to report our findings with regards to the shortcomings of existing methods so as to encourage the machine learning community to consider enhancements in metrics for assessing quality of synthetic images in the material science domain.

Atomic layer deposition (ALD) is a thin film growth technique that is key for both microelectronics and energy applications. Its step-by-step nature and its integration into fully automated clusters with wafer handling systems make is an ideal tool for AI-driven optimization and discovery. In this work we describe an experimental setup and digital twin of an ALD reactor coupled with in-situ characterization techniques that we have developed as a platform for the development and validation of novel algorithms for self-driving labs. Preliminary results show that it is possible to achieve a 100-fold reduction in the time required to optimize new processes. Finally we share some of the lessons learned during the design and validation of our self-driven thin film growth tool.

This paper introduces MatKG, a novel graph database of key concepts in material science spanning the traditional material-structure-property-processing paradigm. MatKG is autonomously generated through transformer-based, large language models and generates pseudo ontological schema through statistical co-occurrence mapping. At present, MatKG contains over 2 million unique relationship triples derived from 80,000 entities. This allows the curated analysis, querying, and visualization of materials knowledge at unique resolution and scale. Further, Knowledge Graph Embedding models are used to learn embedding representations of nodes in the graph which are used for downstream tasks such as link prediction and entity disambiguation. MatKG allows the rapid dissemination and assimilation of data when used as a knowledge base, while enabling the discovery of new relations when trained as an embedding model.

Liquid-liquid phase separation (LLPS) is a fundamental cellular process that isdriven by self-assembly of intrinsically disordered proteins (IDPs), protein-RNAcomplexes, or other bio-molecular systems which can form liquid droplets. Manynatural materials including silk, elastin, and gels are a result of LLPS and thusrational design of such phase-separating peptides can have transformative impact, from designing new biologically inspired materials (e.g., clothing) to selfcompartmentalized drug-delivery systems for biomedical applications. However,given the intrisinc complexity in the rules governing LLPS, rational design of LLPSundergoing peptides remains challenging. We posit that automation, foundationmodels integrated with reinforcement learning approaches and multiscale molecularsimulations can drive the design of novel peptides that undergo LLPS. We describeour progress towards the goal of end-to-end design of phase separating peptidesby summarizing current work at the Argonne National Laboratory’s AdvancedPhoton Source 8ID-I beamline, where a robotic set up in the laboratory is enabledvia simulation and extensive testing of such bio-materials. Together, our approachenables the design of novel bio-materials that can undergo phase separation underdiverse physiological conditions

In recent years, neural networks that incorporate physical symmetry in their architecture have become indispensable tools for overcoming the scarcity of molecular and material data. However, despite its critical importance in deep learning, the design and selection of the normalization layer has often been treated as a side issue. In this study, we first review the unique challenges that batch normalization (BatchNorm) faces in its application to materials science and provide an overview of alternative normalization layers that can address the unique geometric considerations required by physical systems and tasks. While the challenges are diverse, we find that \emph{geometric-match} of a normalization layer can be achieved by ensuring that the normalization preserves not only invariance and equivariance, but also covariance of the task and dataset. Overall, our survey provides a coherent overview of normalization layers for practitioners and presents open-challenges for further developments.

In many applications of machine learning, like drug discovery and material design, the goal is to generate candidates that simultaneously maximize a set of objectives. As these objectives are often conflicting, there is no single candidate that simultaneously maximizes all objectives, but rather a set of Pareto-optimal candidates where one objective cannot be improved without worsening another. Moreover, these objectives, when considered in practice are often under-specified, making diversity of candidates a key consideration. The existing multi-objective optimization methods focus predominantly on covering the Pareto front, failing to capture diversity in the space of candidates. Motivated by the success of GFlowNets for generation of diverse candidates in a single objective setting, in this paper we consider Multi-Objective GFlowNets (MOGFNs). MOGFNs consist of a Conditional GFlowNet which models a family of single-objective sub-problems derived by decomposing the multi-objective optimization problem. Our work is the first to empirically demonstrate conditional GFlowNets. Through a series of experiments on synthetic as well as practically relevant material design and drug discovery tasks, we empirically demonstrate that MOGFNs outperform existing methods in terms of hypervolume, R2-distance and candidate diversity. We also demonstrate the effectiveness of MOGFNs over existing methods in active learning settings.

Score-based methods, particularly denoising diffusion probabilistic models (DDPMs), have demonstrated impressive improvements to the state-of-the-art (SOTA) in generative modeling. DDPM models and related variants, all broadly categorized under diffusion models, are not only applicable to generating entertaining art but are appealing to a wider variety of applications. In this work, we compare the performance of a diffusion model with a Wasserstein Generative Adversarial Network in generating two-phase microstructures of photovoltaic cells. We demonstrate the diffusion model's performance improvements at generating realistic-looking microstructures, as well as its ability to cover several modes in the target distribution.

Chemical design is often complex, requiring the optimal trade-off between several competing objectives. Multi-objective optimization algorithms are designed to optimally balance multiple objectives, but many chemical design approaches use the naïve weighted sum method, which is not guaranteed to give desired solutions. Here, we rigorously assess the performance of genetic algorithms for inverse molecular design, using more advanced multi-objective methods. Chimera and Hypervolume are assessed against relevant baselines for the optimization of molecules with high logP and high QED score. As a more realistic task, we also simulate a drug design campaign, optimizing for synthetically accessible molecules which bind to the 1OYT protein. We show that both methods achieve better formal optimality than the baselines and generate molecules closer to a user-specified Utopian point in property space, mimicking typical materials design objectives.

Piezoresponse force microscopy (PFM) is a scanning microscopy technique that is used to evaluate the nanoscale strain response to an electric voltage applied to the surface of a ferroelectric material. PFM is a powerful tool for imaging, manipulation, and studying the nanoscale functional response of ferroelectric materials, which has been extensively used as a first pass test for ferroelectricity in novel materials with unknown functional properties. However, low signal-to-noise ratio observations arising from the loss of electromechanical signal during polarization switching often result in unreliable information extraction at these observations, hampering our understanding of the material characteristics. To address this challenge, we propose an information recovery framework utilizing subspace-based matrix completion to achieve improved characterization from PFM data. It enables us to efficiently recover and extract reliable information from the data, assisting the modeling efforts for PFM and providing insights for characterization and experimentation practices.

Multimodal characterization is commonly required for understanding materials. User facilities possess the infrastructure to perform these measurements, albeit in serial over days to months. In this paper, we describe a unified multimodal measurement of a single sample library at distant instruments, driven by a concert of distributed agents that use analysis from each modality to inform the direction of the other in real time. Powered by the Bluesky project at the National Synchrotron Light Source II, this experiment is a world's first for beamline science, and provides a blueprint for future approaches to multimodal and multifidelity experiments at user facilities.

Constraint active search is a promising sample-efficient multiobjective experimental design formulation that aims to aid scientists and engineers in searching for new materials. In this proposal, we extend this formulation to situations where one can obtain observations from multiple sources each with a given cost, such as when both computer simulations and a laboratory experiments can be used to calculate (or estimate) properties of a material of interest. We present a novel cost-efficient policy that balances the cost of obtaining observations with the benefit of evaluating a more expensive-to-compute source. Initial results on a synthetic problem show that our proposed methodology is more selective when searching for the most expensive source.

Mitigating the climate crisis requires a rapid transition towards lower carbon energy. Catalyst materials play a crucial role in the electrochemical reactions involved in this transition, such as electrofuel synthesis, renewable fertiliser production and energy storage. In this context, there is a need to discover more effective catalysts for these reactions. Machine learning (ML) holds the potential to efficiently model the properties of materials from large amounts of data, and thus to accelerate electrocatalyst design. The Open Catalyst Project OC20 data set was constructed to that end. However, most existing ML models trained on it are still neither scalable nor accurate enough for practical applications. Here, we propose several task-specific innovations, applicable to most architectures, which increase both computational efficiency and precision. In particular, we aim to improve (1) the graph creation step, (2) atom representations and (3) the energy prediction head. We describe and evaluate these contributions across several architectures, showing up to 5$\times$ reduction in inference time without sacrificing accuracy.

We aim to develop an AI-guided autonomous materials design approach to discover high-performance organic photovoltaics (OPVs). Autonomous synthesis, automated characterization, and AI-based methods will be integrated into a closed-loop approach to drive molecular discovery guided by target criteria for OPV performance: efficiency and stability. The long-term goal of the project is two-fold: (1)in terms of fundamental science, we aim to fill key knowledge gaps in understanding how molecular structure determines OPV stability and efficiency, and advance the science of closed-loop autonomous discovery by learning how to synergistically integrate AI, automated synthesis, and automated testing. (2)In terms of technology, we aim to meet the “10-10” target (10\% efficiency and 10-year stability for OPV materials) to make OPVs a commercial reality for next-generation energy capture applications and for mitigating climate change.

We introduce a task guidance framework for manufacturing settings aiming to improve the well-being and productivity of manufacturing workers completing a given task. The assistive technology proposed in this work centers on a dialogue system built upon semantic frame extraction of process specifications detailing a given manufacturing process. The dialogue system interacts with the technician performing the task by capturing their actions and assisting them in performing relevant steps. Specifically, we develop components to parse expert-authored natural language documents called specs and utilize the parse for task guidance and continual learning. While still in the early stages, we believe that an interactive, assistive AI framework similar to the one we are exploring will become an important component of high-volume manufacturing in the future. Such a system could increase the quality and scalability of next-generation materials and materials-related products, such as batteries or fuel cells, produced by automated materials synthesis techniques and analyzed by automated materials characterization techniques.

Pre-trained large language models (LLMs) are a powerful platform for building custom models for various applications.They have also found success in chemistry, but typically need to be pre-trained on large chemistry datasets such as reaction databases or protein sequences.In this work, we analyze whether one of the largest pre-trained LLMs, GPT-3, can be directly used for chemistry applications by fine-tuning on only a few data points from a chemistry dataset, i.e., without pre-training on a chemistry-specific dataset.We show that GPT-3 can achieve performance competing with baselines on three case studies (polymers, metal-organic frameworks, photoswitches) with representations as simple as the chemical name in both classification and regression settings.Moreover, we demonstrate that GPT-3 can also be fine-tuned for use in inverse design tasks, i.e., to generate a molecule that has properties as specified in a prompt.

Bayesian optimization (BO) has proven to be effective approach for guiding sample-efficient exploration of materials domains and is increasingly being used in automated materials optimization set-ups. However, when exploring novel materials, sample quality may vary unexpectedly, which can even invalidate the optimization procedure if it remains undetected. This issue limits the use of highly-automated optimization loops, especially in high-dimensional materials spaces with a lot of samples. Sample quality may be hard to define unequivocally for a machine but human scientists are usually good at judging sample quality, at least on a cursory yet often sufficient level. In this work, we demonstrate that humans can be added into the BO loop as experts to comment on the sample quality, which results in more trustworthy BO results. We implemented human-in-the-loop BO via a data fusion approach and applied virtual BO cycles on experimental perovskite film stability data from literature. The human-in-the-loop approach facilitates automated materials design and characterization by reducing the occurrence of invalid optimization results.

Transfer of recent advances in deep reinforcement learning to real-worldapplications is hindered by high data demands and thus low efficiency andscalability. Through independent improvements of components such as replaybuffers or more stable learning algorithms, and through massively distributedsystems, training time could be reduced from several days to several hours forstandard benchmark tasks. However, while rewards in simulated environments arewell-defined and easy to compute, reward evaluation becomes the bottleneck inmany real-world environments, e.g. in molecular optimization tasks, wherecomputationally demanding simulations or even experiments are required toevaluate states and to quantify rewards. Therefore, training might becomeprohibitively expensive without an extensive amount of computational resourcesand time. We propose to alleviate this problem by replacing costly ground-truthrewards with rewards modeled by neural networks, counteracting non-stationarityof state and reward distributions during training with an active learning component.We demonstrate that using our proposed ACRL method (actively learning costlyrewards for reinforcement learning), it is possible to train agents in complexreal-world environments orders of magnitudes faster. By enabling the applicationof reinforcement learning methods to new domains, we show that we can findinteresting and non-trivial solutions to real-world optimization problems inchemistry, materials science and engineering.

A major obstacle to the realization of novel inorganic materials with desirable properties is the ability to perform efficient optimization across both materials properties and synthesis of those materials. In this work, we propose a reinforcement learning (RL) approach to inverse inorganic materials design, which can identify promising compounds with specified properties and synthesizability constraints. Our model learns chemical guidelines such as charge and electronegativity neutrality while maintaining chemical diversity and uniqueness. We demonstrate a multi-objective RL approach, which can generate novel compounds with targeted materials properties including formation energy and bulk/shear modulus alongside a lower sintering temperature synthesis objectives. Using this approach, the model can predict promising compounds of interest, while informing an optimized chemical design space for inorganic materials discovery.

Besides increasing trust in the human-AI relationship, XAI methods have the potential to promote new scientific insight. Graph neural networks (GNNs) have recently established themselves as a valuable tool in chemistry and material sciences. Various XAI methods have already been applied to gain new understanding of real-world scientific questions in these application domains. To that end we propose MEGAN, a multi-explanation graph attention network. Unlike common post-hoc XAI methods, our model is self-explaining and features multiple explanation channels, which can be chosen independent of the task specifications. We first validate our model on a synthetic graph regression dataset. We then apply our model to the prediction of water solubility for chemical compounds. We find that it learns to produce explanations consistent with human intuition, opening the way to learning from our model in less well-understood tasks.

One of the most important problems in rational design of batteries is predicting the properties of the Solid Electrolyte Interphase, which (for a metallic anode) is the part of the battery where metallic and non-metallic components come into contact. However, there is a fundamental problem with predicting the properties of such a mixed material: the two components are best simulated with incompatible levels of density functional theory. Pure functionals perform well for metallic properties, while hybrid or long-range-corrected density functionals perform better for molecular properties and reaction barriers. We demonstrate a simple method to obviate this conflict by training a machine learning potential energy surface using both levels of theory via transfer learning. We further show that the resulting model is more accurate than models trained individually to these levels of theory, allowing more accurate property prediction and potentially faster materials discovery.

Accurate uncertainty estimates can significantly improve the performance of iterative design of experiments, as in Sequential and Reinforcement learning. For many such problems in engineering and the physical sciences, the design task depends on multiple correlated model outputs as objectives and/or constraints. To better solve these problems, we propose a recalibrated bootstrap method to generate multivariate prediction intervals for bagged models and show that it is well-calibrated. We apply the recalibrated bootstrap to a simulated sequential learning problem with multiple objectives and show that it leads to a marked decrease in the number of iterations required to find a satisfactory candidate. This indicates that the recalibrated bootstrap could be a valuable tool for practitioners using machine learning to optimize systems with multiple competing targets.

Structure solution of nanostructured materials that have limited long-range order remains a bottleneck in materials development. We present a deep learning algorithm, DeepStruc, that can solve a simple nanoparticle structure directly from a Pair Distribution Function (PDF) obtained from total scattering data by using a conditional variational autoencoder. We first apply DeepStruc to PDFs from seven different structure types of monometallic nanoparticles, and show that structures can be solved from both simulated and experimental PDFs, including PDFs from nanoparticles that are not present in the training distribution. We also apply DeepStruc to a system of hcp, fcc and stacking faulted nanoparticles, where DeepStruc recognizes stacking faulted nanoparticles as an interpolation between hcp and fcc nanoparticles and is able to solve stacking faulted structures from PDFs. Our findings suggests that DeepStruc is a step towards a general approach for structure solution of nanomaterials.

Microstructures, i.e., architected materials, are designed today, typically, by maximizing an objective, such as bulk modulus, subject to a volume constraint. However, in many applications, it is often more appropriate to impose constraints on other physical quantities of interest.In this paper, we consider such generalized microstructural optimization problems where any of the microstructural quantities, namely, bulk, shear, Poisson ratio, or volume, can serve as the objective, while the remaining can serve as constraints. In particular, we propose here a neural-network (NN) framework to solve such problems. The framework relies on the classic density formulation of microstructural optimization, but the density field is represented through the NN's weights and biases.The main characteristics of the proposed NN framework are: (1) it supports automatic differentiation, eliminating the need for manual sensitivity derivations, (2) smoothing filters are not required due to implicit filtering, (3) the framework can be easily extended to multiple-materials, and (4) a high-resolution microstructural topology can be recovered through a simple post-processing step. The framework is illustrated through a variety of microstructural optimization problems.

Recent work has shown the potential of graph neural networks to efficiently predict material properties, enabling high-throughput screening of materials. Training these models, however, often requires large quantities of labelled data, obtained via costly methods such as ab initio calculations or experimental evaluation. By leveraging a series of material-specific transformations, we introduce CrystalCLR, a framework for constrastive learning of representations with crystal graph neural networks. With the addition of a novel loss function, our framework is able to learn representations competitive with engineered fingerprinting methods. We also demonstrate that via model finetuning, contrastive pretraining can improve the performance of graph neural networks for prediction of material properties and significantly outperform traditional ML models that use engineered fingerprints. Lastly, we observe that CrystalCLR produces material representations that form clusters by compound class.

The design of functional molecules relies on the representation used: a flexible and informative representation can improve downstream generation tasks. String representations such as SMILES and SELFIES serve as the basis for chemical language models, and the robustness of SELFIES makes it naturally suited for molecular optimization with genetic algorithms. But while SMILES and SELFIES are atomic representations, several recent approaches take advantage of the inductive bias of molecular fragments. In this work, we present Group SELFIES, introducing group tokens that represent functional groups or entire substructures while maintaining robustness. Group tokens give control over which structures should be preserved during optimization. Experiments indicate that Group SELFIES improves distribution learning and improves the quality of molecules generated by simply taking random Group SELFIES strings. The code is available at \url{https://anonymous.4open.science/r/group-selfies-4D87/}.

Characterization of material structure with X-ray or neutron scattering using e.g. Pair Distribution Function (PDF) analysis most often rely on refining a structure model against an experimental dataset. However, identifying a suitable model is often a bottleneck. Recently, new automated approaches have made it possible to test thousands of models for each dataset, but these methods are computationally expensive, and analysing the output, i.e., extracting structural information from the resulting fits in a meaningful way is challenging. Our Machine Learning based Motif Extractor (ML-MotEx) trains an ML algorithm on thousands of fits, and uses SHAP (SHapley Additive exPlanation) values to identify which model features are important for the fit quality. We use the method for 4 different chemical systems including disordered nanomaterials and clusters. ML-MotEx opens for a new type of modelling where each feature in a model is assigned an importance value for the fit quality based on explainable ML.

Representing crystal structures of materials in a form preferable for neural networks is crucial for enabling machine learning applications involving the estimation of crystal structures. This paper proposes Neural Structure Fields (NeSF) as an accurate and practical approach to representing crystal structures by neural networks. Our crucial idea, inspired by the concepts of both vector fields in physics and implicit neural representations in computer vision, is to consider a crystal structure as a continuous field rather than a discrete set of atoms. Unlike existing grid-based discretized spatial representations, NeSF is free from a trade-off between spatial resolution and computational complexity and can represent complex crystal structures. We demonstrate the expressibility of NeSF successfully in applying it to auto-encoding of crystal structures.

The proliferation of deep learning (DL) techniques in recent years has often resulted in the creation of progressively larger datasets and deep learning architectures. As the expressive power of DL models has grown, so has the compute capacity needed to effectively train the models. One such example is the OpenCatalyst dataset in the emerging field of scientific machine learning, which has elevated the compute requirements needed to effectively train graph neural networks (GNNs) on complex scientific data. The extensive compute complexity involved in training GNNs on the OpenCatalyst dataset makes it very costly to perform hyperparameter optimization (HPO) using traditional methods, such as grid search or even Bayesian optimization-based approaches. Given this challenge, we propose a novel methodology for effective, cost-aware HPO on GNN training on OpenCatalyst that leverages a multi-fidelity approach with experiments on reduced datasets, hyperparameter importance, and computational budget considerations. We show speed ups by over 50 percent when performing hyperparameter optimization of the E(n)-GNN model on the OpenCatalyst dataset.

Most chemical reactions result in numerous by-products and side-products, apart from the intended major product. While chemists can predict many of the main process impurities, it remains a challenge to enumerate the possible minor impurities and even more of a challenge to systematically predict and track impurities derived from raw materials or those that have propagated from one synthetic step to the next. In this study, we developed an AI-assisted approach to predict and track impurities across multi-step reactions using the main reactants, and optionally reagents, solvents and impurities in these materials, as input. We demonstrated the utility of this tool for a simple case of synthesis of paracetamol from phenol, and provide a generalized framework that covers most chemical reactions. Our solution can be applied to enable (1) faster elucidation of impurities, (2) automated interpretation of data generated from high-throughput reaction screening, and (3) more thorough raw materials risk assessments, with each of these representing key workflows in small molecule drug substance commercial process development.

We introduce a novel approach to conformer search, the discovery of three-dimensional structures for two-dimensional molecular formulas. We focus on organic molecules using deep imitation learning and equivariant graph neural networks, with the prospect of using reinforcement learning algorithms for fine tuning. To that end, we present our interactive environment that describes the molecule in a ridig-rotor approximation and leverage a behavioral cloning torsion policy to autoregressively determine the dihedral angles of the molecule ultimately yielding a three-dimensional molecular structure. For our policy architecture, we leverage an SE(3) equivariant neural network, which enables us to exploit inherent molecular symmetries and to respect the topology of the angle distribution using a Mixture of Projected Normals action distribution. Our preliminary results for a policy trained on a behavioral cloning objective using the QM9 dataset for expert trajectories shows that the policy can accurately predict torsion angles for various molecules. We believe this to be a promising starting point for future work pertaining to performing conformer search using deep reinforcement learning.

Insect-borne diseases kill >0.5 million people annually. Currently available repellents for personal or household protection are limited in their efficacy, applicability, and safety profile. Here, we describe a machine-learning-driven high-throughput method for the discovery of novel repellent molecules. To achieve this, we digitized a large, historic dataset containing ~19,000 mosquito repellency measurements. We then trained a graph neural network (GNN) to map molecular structure and repellency. We applied this model to select 317 candidate molecules to test in parallelizable behavioral assays, quantifying repellency in multiple pest species and in follow-up trials with human volunteers. The GNN approach outperformed a chemoinformatics model and produced a hit rate that increased with training data size, suggesting that both model innovation and novel data collection were integral to predictive accuracy. We identified >10 molecules with repellency similar to or greater than the most widely used repellents. This approach enables computational screening of billions of possible molecules to identify empirically tractable numbers of candidate repellents, leading to accelerated progress towards solving a global health challenge.