Timezone: »

Workshop
AI for Accelerated Materials Design (AI4Mat)
Santiago Miret · Marta Skreta · Zamyla Morgan-Chan · Benjamin Sanchez-Lengeling · Shyue Ping Ong · Alan Aspuru-Guzik

Fri Dec 02 06:00 AM -- 03:00 PM (PST) @ Room 386

Self-Driving Materials Laboratories have greatly advanced the automation of material design and discovery. They require the integration of diverse fields and consist of three primary components, which intersect with many AI-related research topics:

- AI-Guided Design. This component intersects heavily with algorithmic research at NeurIPS, including (but not limited to) various topic areas such as: Reinforcement Learning and data-driven modeling of physical phenomena using Neural Networks (e.g. Graph Neural Networks and Machine Learning For Physics).

- Automated Chemical Synthesis. This component intersects significantly with robotics research represented at NeurIPS, and includes several parts of real-world robotic systems such as: managing control systems (e.g. Reinforcement Learning) and different sensor modalities (e.g. Computer Vision), as well as predictive models for various phenomena (e.g. Data-Based Prediction of Chemical Reactions).

- Automated Material Characterization. This component intersects heavily with a diverse set of supervised learning techniques that are well-represented at NeurIPS such as: computer vision for microscopy images and automated machine learning based analysis of data generated from different kinds of instruments (e.g. X-Ray based diffraction data for determining material structure).

 Fri 6:00 a.m. - 6:30 a.m. Opening Remarks (AI4Mat Program Committee) 🔗 Fri 6:30 a.m. - 7:30 a.m. Everyday Research Challenges in AI for Automated Materials Design (Panel) Panelists: Rocio Mercado - Incoming Assistant Professor Chalmers University Katherine Sytwu - Postdoc Lawrence Berkeley National Lab Felix Streith-Kalthoff - Postdoc University of Toronto Tian Xie - Researcher Microsoft AI4Science 🔗 Fri 7:30 a.m. - 8:00 a.m. AI-Guided Design Keynote (Keynote) Elsa Olivetti 🔗 Fri 8:00 a.m. - 8:30 a.m. Coffee Break (Break) 🔗 Fri 8:30 a.m. - 8:40 a.m. Open MatSci ML Toolkit: A Flexible Framework for Machine Learning in Materials Science (Spotlight) []   link » The Open MatSci ML Toolkit is a flexible, self-contained and scalable Python-based framework to apply deep learning models and methods on scientific data with a specific focus on materials science and the OpenCatalyst Dataset. The primary components of our toolkit include: 1.Scalable computation of experiments leveraging PyTorch Lightning across different computation capabilities (laptop, server, cluster) and hardware platforms (CPU, GPU, XPU) without sacrificing performance in the compute and modeling; 2. Support for DGL for rapid graph neural network development. By sharing this toolkit with the research community via open-source release, we aim to: 1. Ease of use for new machine learning researchers and practitioners that want get started on interacting with the OpenCatalyst dataset which currently makes up the largest computational materials science dataset; 2. Enable the scientific community to apply advanced machine learning tools to high-impact scientific challenges, such as modeling of materials behavior for climate change applications. Link » Santiago Miret · Kin Long Kelvin Lee · Carmelo Gonzales · Marcel Nassar · Krzysztof Sadowski 🔗 Fri 8:40 a.m. - 8:50 a.m. Accelerating the Discovery of Rare Materials with Bounded Optimization Techniques (Spotlight) []   link »    Discovering a rare material within a vast search space exhibits a Needle-in-a-Haystack challenge. This challenge of finding a rare material, i.e., the "needle", inside a vast search space, i.e., the "haystack", arises when there is an extreme imbalance of optimum conditions relative to the size of the search space. For example, only 0.82% out of 146k total materials in the open-access Materials Project database have a negative Poisson's ratio, a rare material property. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence to a global optimum or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI, that builds on conventional Bayesian optimization principles to quickly and efficiently optimize Needle-in-a-Haystack problems in both less time and fewer experiments by addressing the common convergence and pigeonholing issues. ZoMBI actively extracts knowledge from the previously best-performing evaluated experiments to iteratively zoom in the sampling search bounds towards the global optimum "needle" and then prunes the memory of low-performing historical experiments to accelerate compute times by reducing the algorithm time complexity from $O(n^3)$ to $O(1)$, as the number of experiments sampled increases. Additionally, ZoMBI implements two custom acquisition functions that use active learning to further guide the sampling of new experiments towards the global optimum. We validate the algorithm's performance on two real-world 5-dimensional Needle-in-a-Haystack material property optimization datasets: discovery of auxetic Poisson's ratio materials and discovery of high thermoelectric figure of merit materials. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering materials in under 100 experiments that are up to 3x more highly optimized than those discovered by current state-of-the-art algorithms. Link » Alexander E. Siemenn · zekun ren · Qianxiao Li · Tonio Buonassisi 🔗 Fri 8:50 a.m. - 9:00 a.m. A Data-efficient Multiobjective Machine Learning Method For 3D-printed Architected Materials Design (Spotlight) []   link »    Architected materials that consist of multiple subelements arranged in particular orders can demonstrate a much broader range of properties than their constituent materials. However, the rational design of these materials generally relies on experts’ prior knowledge and requires painstaking effort. Here, we present a data efficient method for the multiproperty optimization of 3D-printed architected materials utilizing a machine learning (ML) cycle consisting of the finite element method (FEM) and 3D neural networks. Specifically, we applied our method to orthopedic implant design. Compared to expert designs, our experience-free method designed microscale heterogeneous architectures with a biocompatible elastic modulus and higher strength. Furthermore, inspired by the knowledge learned by the neural networks, we developed machine-human synergy, adapting the ML-designed architecture to fix a macroscale, irregularly shaped animal bone defect. Such adaptation exhibits 20 % higher experimental load-bearing capacity than the expert design. Thus, our method opens a new paradigm for the fast and intelligent design of architected materials with tailored mechanical, physical, and chemical properties. Link » Bo Peng · Ye Wei · Yu Qin · Jiabao Dai · Liuliu Han · Yue Li · Peng Wen 🔗 Fri 9:00 a.m. - 9:10 a.m. MolPAL: Software for Sample Efficient High-Throughput Virtual Screening (Spotlight) []   link »    Structure-based virtual screening (SBVS) of ultra-large chemical libraries has led to the discovery of novel inhibitors for challenging protein targets. However, screening campaigns of these magnitudes are expensive and thus impractical to employ in standard practice. As the broad goal of most SBVS workflows is the identification of the most potent compounds in the library, the task can be viewed as an optimization problem. Previous work has demonstrated the ability for Bayesian optimization to improve sample efficiency in SBVS using the MolPAL software. In this tutorial, we provide a broad algorithmic overview of the MolPAL software and a guide for its utilization in a prospective virtual screening task. Link » David Graff · Connor Coley 🔗 Fri 9:10 a.m. - 9:30 a.m. AI-Guided Design Spotlight Q&A (Q&A) 🔗 Fri 9:30 a.m. - 10:30 a.m. Poster Session 🔗 Fri 10:30 a.m. - 11:00 a.m. Lunch 🔗 Fri 11:00 a.m. - 11:30 a.m. Automated Materials Synthesis Keynote (Keynote) Abstract: The ability to quickly iterate through design cycles relies on our ability to synthesize molecules and materials found in virtual libraries or proposed by generative models. In this talk, we will focus on a primary consideration of molecular design workflows: the chemical space that comprises the search space for a molecular screening/optimization campaign. That is, the manner in which the search is constrained to a finite library of molecules or, following an increasingly popular trend, the manner in which the search navigates a virtually infinite space of molecules. We will talk about how models to predict chemical reactivity inform our ability to define and navigate these spaces. Further, we will discuss the challenge of sample efficiency, as most algorithms for molecular design operate in a regime that is severely misaligned with the reality of experimental work in terms of the number of candidates that can be evaluated within a practical budget. Bio: Connor W. Coley is an Assistant Professor at MIT in the Department of Chemical Engineering and the Department of Electrical Engineering and Computer Science. He received his B.S. and Ph.D. in Chemical Engineering from Caltech and MIT, respectively, and did his postdoctoral training at the Broad Institute. His research group at MIT develops new methods at the intersection of data science, chemistry, and laboratory automation to streamline discovery in the chemical sciences with an emphasis on therapeutic discovery. Key research areas in the group include the design of new neural models for representation learning on molecules, data-driven synthesis planning, in silico strategies for predicting the outcomes of organic reactions, model-guided Bayesian optimization, and de novo molecular generation. Connor is a recipient of C&EN’s “Talented Twelve” award, Forbes Magazine’s “30 Under 30” for Healthcare, the NSF CAREER award, and the Bayer Early Excellence in Science Award. Outside of MIT, Connor serves as an advisor to both early- and late-stage companies including Entos, Revela, Galixir, Kebotix, Anagenex, and Dow. Connor Coley 🔗 Fri 11:30 a.m. - 11:40 a.m. Element-Wise Formulation of Inorganic Retrosynthesis (Spotlight) []   link »    Synthesizing new inorganic functional materials is a practical goal of materials science. While the advances in computational techniques accelerated the virtual design, the actual synthesis of predicted candidate materials still remain as an expensive and slow process. While a few initial studies attempted to predict the synthesis routes for inorganic crystals, the existing models do not yield the uncertainty of the predictions and could produce thermodynamically unrealistic precursor chemicals. Here, we propose an element-wise graph neural network to predict the inorganic synthesis recipes. The trained model outperforms the popularity-based statistical baseline model for top-k exact match accuracy test, showing the validity of our approach for inorganic solid-state synthesis. We further validate our model by the publication-year-split test, where the model trained based on the materials data until the year 2016 is shown to successfully predict the synthetic precursors for the materials synthesized after 2016. The high correlation between the classification score and prediction accuracy suggests that the prediction score can be interpreted as a measure of uncertainty. Link » Seongmin Kim · Juhwan Noh · Geun Ho Gu · SHU-AN CHEN · Yousung Jung 🔗 Fri 11:40 a.m. - 11:50 a.m. A High-Throughput Platform for Efficient Exploration of Polypeptides Chemical Space via Automation and Machine Learning (Spotlight) []   link »    Rapid and in-depth exploration of the chemical space of high molecular weight synthetic polypeptides via the ring-opening polymerization (ROP) of N-carboxyanhydride (NCA) is a viable approach towards protein mimics and functional biomaterials. Here, we develop an efficient chemistry for the high throughput diversification of polypeptides based on a click-like reaction between selenolate and various electrophiles in aqueous solutions. With the assistance of automation and machine learning, iterative exploration of the random heteropolypeptides (RHPs) library efficiently and effectively identifies hit materials from a model system of which we have little prior knowledge. This automated and high-throughput platform provides a useful interface between wet and dry experiment, which would accelerate the discovery of new polypeptide materials for unmet challenges such as de novo design of artificial enzyme, biomacromolecule delivery, and understanding of intrinsically disordered proteins. Link » Guangqi Wu · Connor Coley · Hua Lu 🔗 Fri 11:50 a.m. - 12:00 p.m. Differential top-k learning for template-based single-step retrosynthesis (Spotlight) []   link »    Retrosynthesis is one of the core tasks in the organic molecule design cycle, yet it is still a computational challenge to produce suitable sets of precursors for a desired product. Commonly used template-based approaches reduce the problem to a multi-class classification task for single steps. However, reactions in available datasets are noisy and incomplete, making usual training methods problematic. In this work, considering that multiple disconnections are possible for a product, we propose training models using differential top-k losses. We show that using these loss functions yields improvements in every top-N metric, with little overhead relative to cross-entropy. The use of more powerful models, more diverse and complete datasets, and other methodologies, is expected to yield significant improvements on this task when combined with the training approach presented here. Link » Andres M Bran · Philippe Schwaller 🔗 Fri 12:00 p.m. - 12:10 p.m. A self-driving laboratory optimizes a scalable materials manufacturing process (Spotlight) []   link »    Solution-based coating methods offer a low-cost method for depositing coatings at scale. It is difficult, however, to obtain high quality coatings using these methods due to the complex physical phenomena at play. Here, we show how a self-driving laboratory can optimize a scalable spray-coating process applicable to manufacturing diverse clean-energy technologies. We demonstrate this system by optimizing a spray-combustion process for depositing conductive palladium films. This optimization yielded films with conductivities of 4.08 MS/m, doubling the state-of-the art film conductivity possible with this process and rivaling Pd film conductivities obtained using vacuum-based sputtering processes (2 to 6 MS/m). The rich data gathered by the self-driving laboratory also provides mechanistic insights into the coating process. The champion coating conditions were scaled up to an 8× larger area using the same spray-coating apparatus with no further optimization and no reduction in coating quality. This work shows how self-driving laboratories can optimize spray-coating for depositing coatings at scale. Link » Connor Rupnow · Benjamin MacLeod · Mehrdad Mokhtari · Karry Ocean · Kevan Dettelbach · Daniel Lin · Fraser Parlane · Hsi Chiu · Michael Rooney · Christopher Waizenegger · Elija de Hoog · Curtis Berlinguette 🔗 Fri 12:10 p.m. - 12:30 p.m. Automated Materials Synthesis Spotlight Q&A (Q&A) 🔗 Fri 12:30 p.m. - 1:00 p.m. Automated Materials Characterization Keynote (Keynote) Abstract: Deep learning schemes have already impacted areas such as cognitive game theory (e.g., computer chess and the game of Go), pattern (e.g., facial or fingerprint) recognition, event forecasting, and bioinformatics. They are beginning to make major inroads within physics, chemistry and materials sciences and hold considerable promise for accelerating the discovery of new theories and materials. In this talk, I will introduce deep convolutional neural networks and how they can be applied to the computer vision problems in transmission electron microscopy and tomographic imaging. Bio: Huolin Xin is a full professor at UC Irvine. He graduated from the Physics Department of Cornell University in 2011 and joined the University of California, Irvine in 2018. Prior to becoming a professor at UCI, he worked at Brookhaven National Laboratory as a scientific staff member and a principal investigator from 2013 to 2018. His research has resulted in more than 280 peer-reviewed publications (h-index 73 and citations 24,600). He received the MRS Oustanding Early Career Investigator Award, MSA Burton Medal, DOE Early Career Award, and the UCI Distinguished Early-Career Faculty for Research in 2020. He was the Chair of the largest international electron microscopy conference, Microscopy and Microanalysis, in 2020. His work on battery materials has been selected as the 2020, 2019 and 2014's Top-10 Scientific Achievements by Brookhaven Lab. Huolin Xin 🔗 Fri 1:00 p.m. - 1:30 p.m. Coffee Break (Break) 🔗 Fri 1:30 p.m. - 1:40 p.m. Robust design of semi-automated clustering models for 4D-STEM datasets (Spotlight) []   link »    Materials discovery and design require characterizing material structure at the nanometer and sub-nanometer scale. Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM) resolves the crystal structure of materials, but many 4D-STEM data analysis pipelines are not suited for identification of anomalous and unexpected structures. This work introduces improvements to the iterative Non-Negative Matrix Factorization (NMF) method by implementing consensus clustering for ensemble learning. We evaluate the performance of models during parameter tuning and find that consensus clustering improves performance in all cases and is able to recover specific grains missed by the best performing model in the ensemble. The methods introduced in this work can be applied broadly to materials characterization datasets to aid in the design of new materials. Link » Alexandra Bruefach · Colin Ophus · Mary Scott 🔗 Fri 1:40 p.m. - 1:50 p.m. A Survey on Evaluation Metrics for Synthetic Material Micro-Structure Images from Generative Models (Spotlight) []   link »    The evaluation of synthetic micro-structure images is an emerging problem as machine learning and materials science research have evolved together. Typical state of the art methods in evaluating synthetic images from generative models have relied on the Frechet Inception Distance. However, this and other similar methods, are limited in the materials domain due to the unique features that characterize physically accurate micro-structures and limited dataset sizes. In this study we evaluate a variety of methods on scanning electron microscope (SEM) images of graphene-reinforced polyurethane foams. The primary objective of this paper is to report our findings with regards to the shortcomings of existing methods so as to encourage the machine learning community to consider enhancements in metrics for assessing quality of synthetic images in the material science domain. Link » Devesh Shah · Anirudh Suresh · Alemayehu Solomon Admasu · devesh upadhyay · Kalyanmoy Deb 🔗 Fri 1:50 p.m. - 2:00 p.m. Experimental platform and digital twin for AI-driven materials optimization and discovery for microelectronics using atomic layer deposition (Spotlight) []   link » Atomic layer deposition (ALD) is a thin film growth technique that is key for both microelectronics and energy applications. Its step-by-step nature and its integration into fully automated clusters with wafer handling systems make is an ideal tool for AI-driven optimization and discovery. In this work we describe an experimental setup and digital twin of an ALD reactor coupled with in-situ characterization techniques that we have developed as a platform for the development and validation of novel algorithms for self-driving labs. Preliminary results show that it is possible to achieve a 100-fold reduction in the time required to optimize new processes. Finally we share some of the lessons learned during the design and validation of our self-driven thin film growth tool. Link » Angel Yanguas-Gil · Steve Letourneau · Noah Paulson · Jeffrey Elam 🔗 Fri 2:00 p.m. - 2:10 p.m. The Largest Knowledge Graph in Materials Science - Entities, Relations, and Link Prediction through Graph Representation Learning (Spotlight) []   link »    This paper introduces MatKG, a novel graph database of key concepts in material science spanning the traditional material-structure-property-processing paradigm. MatKG is autonomously generated through transformer-based, large language models and generates pseudo ontological schema through statistical co-occurrence mapping. At present, MatKG contains over 2 million unique relationship triples derived from 80,000 entities. This allows the curated analysis, querying, and visualization of materials knowledge at unique resolution and scale. Further, Knowledge Graph Embedding models are used to learn embedding representations of nodes in the graph which are used for downstream tasks such as link prediction and entity disambiguation. MatKG allows the rapid dissemination and assimilation of data when used as a knowledge base, while enabling the discovery of new relations when trained as an embedding model. Link » Vineeth Venugopal · Sumit Pai · Elsa Olivetti 🔗 Fri 2:10 p.m. - 2:30 p.m. Automated Materials Synthesis Spotlight Q&A (Q&A) 🔗 Fri 2:30 p.m. - 3:00 p.m. Closing Remarks (AI4Mat Program Committee) 🔗 - Integrating AI, automation and multiscale simulations for end-to-end design of phase-separating proteins (Poster) []  []   link » Liquid-liquid phase separation (LLPS) is a fundamental cellular process that isdriven by self-assembly of intrinsically disordered proteins (IDPs), protein-RNAcomplexes, or other bio-molecular systems which can form liquid droplets. Manynatural materials including silk, elastin, and gels are a result of LLPS and thusrational design of such phase-separating peptides can have transformative impact, from designing new biologically inspired materials (e.g., clothing) to selfcompartmentalized drug-delivery systems for biomedical applications. However,given the intrisinc complexity in the rules governing LLPS, rational design of LLPSundergoing peptides remains challenging. We posit that automation, foundationmodels integrated with reinforcement learning approaches and multiscale molecularsimulations can drive the design of novel peptides that undergo LLPS. We describeour progress towards the goal of end-to-end design of phase separating peptidesby summarizing current work at the Argonne National Laboratory’s AdvancedPhoton Source 8ID-I beamline, where a robotic set up in the laboratory is enabledvia simulation and extensive testing of such bio-materials. Together, our approachenables the design of novel bio-materials that can undergo phase separation underdiverse physiological conditions Link » Arvind Ramanathan 🔗 - Geometric Considerations for Normalization Layers in Equivariant Neural Networks (Poster) []  []   link » In recent years, neural networks that incorporate physical symmetry in their architecture have become indispensable tools for overcoming the scarcity of molecular and material data. However, despite its critical importance in deep learning, the design and selection of the normalization layer has often been treated as a side issue. In this study, we first review the unique challenges that batch normalization (BatchNorm) faces in its application to materials science and provide an overview of alternative normalization layers that can address the unique geometric considerations required by physical systems and tasks. While the challenges are diverse, we find that \emph{geometric-match} of a normalization layer can be achieved by ensuring that the normalization preserves not only invariance and equivariance, but also covariance of the task and dataset. Overall, our survey provides a coherent overview of normalization layers for practitioners and presents open-challenges for further developments. Link » Max Aalto · Ekdeep S Lubana · Hidenori Tanaka 🔗 - Multi-Objective GFlowNets (Poster) []   link » In many applications of machine learning, like drug discovery and material design, the goal is to generate candidates that simultaneously maximize a set of objectives. As these objectives are often conflicting, there is no single candidate that simultaneously maximizes all objectives, but rather a set of Pareto-optimal candidates where one objective cannot be improved without worsening another. Moreover, these objectives, when considered in practice are often under-specified, making diversity of candidates a key consideration. The existing multi-objective optimization methods focus predominantly on covering the Pareto front, failing to capture diversity in the space of candidates. Motivated by the success of GFlowNets for generation of diverse candidates in a single objective setting, in this paper we consider Multi-Objective GFlowNets (MOGFNs). MOGFNs consist of a Conditional GFlowNet which models a family of single-objective sub-problems derived by decomposing the multi-objective optimization problem. Our work is the first to empirically demonstrate conditional GFlowNets. Through a series of experiments on synthetic as well as practically relevant material design and drug discovery tasks, we empirically demonstrate that MOGFNs outperform existing methods in terms of hypervolume, R2-distance and candidate diversity. We also demonstrate the effectiveness of MOGFNs over existing methods in active learning settings. Link » Moksh Jain · Sharath Chandra Raparthy · Alex Hernandez-Garcia · Jarrid Rector-Brooks · Yoshua Bengio · Santiago Miret · Emmanuel Bengio 🔗 - Generative Design of Material Microstructures for Organic Solar Cells using Diffusion Models (Poster) []  []   link » Score-based methods, particularly denoising diffusion probabilistic models (DDPMs), have demonstrated impressive improvements to the state-of-the-art (SOTA) in generative modeling. DDPM models and related variants, all broadly categorized under diffusion models, are not only applicable to generating entertaining art but are appealing to a wider variety of applications. In this work, we compare the performance of a diffusion model with a Wasserstein Generative Adversarial Network in generating two-phase microstructures of photovoltaic cells. We demonstrate the diffusion model's performance improvements at generating realistic-looking microstructures, as well as its ability to cover several modes in the target distribution. Link » Ethan Herron · Xian Yeow Lee · Aditya Balu · Baskar Ganapathysubramanian · Soumik Sarkar · Adarsh Krishnamurthy 🔗 - Assessing multi-objective optimization of molecules with genetic algorithms against relevant baselines (Poster) []  []   link » Chemical design is often complex, requiring the optimal trade-off between several competing objectives. Multi-objective optimization algorithms are designed to optimally balance multiple objectives, but many chemical design approaches use the naïve weighted sum method, which is not guaranteed to give desired solutions. Here, we rigorously assess the performance of genetic algorithms for inverse molecular design, using more advanced multi-objective methods. Chimera and Hypervolume are assessed against relevant baselines for the optimization of molecules with high logP and high QED score. As a more realistic task, we also simulate a drug design campaign, optimizing for synthetically accessible molecules which bind to the 1OYT protein. We show that both methods achieve better formal optimality than the baselines and generate molecules closer to a user-specified Utopian point in property space, mimicking typical materials design objectives. Link » Nathanael Kusanda · Gary Tom · Riley Hickman · AkshatKumar Nigam · Kjell Jorner · Alan Aspuru-Guzik 🔗 - Information Recovery via Matrix Completion for Piezoresponse Force Microscopy Data (Poster) []  []   link » Piezoresponse force microscopy (PFM) is a scanning microscopy technique that is used to evaluate the nanoscale strain response to an electric voltage applied to the surface of a ferroelectric material. PFM is a powerful tool for imaging, manipulation, and studying the nanoscale functional response of ferroelectric materials, which has been extensively used as a first pass test for ferroelectricity in novel materials with unknown functional properties. However, low signal-to-noise ratio observations arising from the loss of electromechanical signal during polarization switching often result in unreliable information extraction at these observations, hampering our understanding of the material characteristics. To address this challenge, we propose an information recovery framework utilizing subspace-based matrix completion to achieve improved characterization from PFM data. It enables us to efficiently recover and extract reliable information from the data, assisting the modeling efforts for PFM and providing insights for characterization and experimentation practices. Link » Henry Yuchi · Kerisha Williams · Gardy Ligonde · Matthew Repasky · Yao Xie · Nazanin Bassiri-Gharb 🔗 - Self-driving Multimodal Studies at User Facilities (Poster) []  []   link » Multimodal characterization is commonly required for understanding materials. User facilities possess the infrastructure to perform these measurements, albeit in serial over days to months. In this paper, we describe a unified multimodal measurement of a single sample library at distant instruments, driven by a concert of distributed agents that use analysis from each modality to inform the direction of the other in real time. Powered by the Bluesky project at the National Synchrotron Light Source II, this experiment is a world's first for beamline science, and provides a blueprint for future approaches to multimodal and multifidelity experiments at user facilities. Link » Phillip M Maffettone · Daniel Allan · Stuart Campbell · Matthew Carbone · Thomas Caswell · Brian DeCost · Dmitri Gavrilov · Marcus Hanwell · Howie Joress · Joshua Lynch · Bruce Ravel · Stuart Wilkins · Jakub Wlodek · Daniel Olds 🔗 - On Multi-information source Constraint Active Search (Poster) []  []   link » Constraint active search is a promising sample-efficient multiobjective experimental design formulation that aims to aid scientists and engineers in searching for new materials. In this proposal, we extend this formulation to situations where one can obtain observations from multiple sources each with a given cost, such as when both computer simulations and a laboratory experiments can be used to calculate (or estimate) properties of a material of interest. We present a novel cost-efficient policy that balances the cost of obtaining observations with the benefit of evaluating a more expensive-to-compute source. Initial results on a synthetic problem show that our proposed methodology is more selective when searching for the most expensive source. Link » Gustavo Malkomes · Bolong Cheng · Santiago Miret 🔗 - PhAST: Physics-Aware, Scalable, and Task-specific GNNs for accelerated catalyst design (Poster) []   link » Mitigating the climate crisis requires a rapid transition towards lower carbon energy. Catalyst materials play a crucial role in the electrochemical reactions involved in this transition, such as electrofuel synthesis, renewable fertiliser production and energy storage. In this context, there is a need to discover more effective catalysts for these reactions. Machine learning (ML) holds the potential to efficiently model the properties of materials from large amounts of data, and thus to accelerate electrocatalyst design. The Open Catalyst Project OC20 data set was constructed to that end. However, most existing ML models trained on it are still neither scalable nor accurate enough for practical applications. Here, we propose several task-specific innovations, applicable to most architectures, which increase both computational efficiency and precision. In particular, we aim to improve (1) the graph creation step, (2) atom representations and (3) the energy prediction head. We describe and evaluate these contributions across several architectures, showing up to 5$\times$ reduction in inference time without sacrificing accuracy. Link » ALEXANDRE DUVAL · Victor Schmidt · Alex Hernandez-Garcia · Santiago Miret · Yoshua Bengio · David Rolnick 🔗 - Autonomous Materials Discovery for Organic Photovoltaics (Poster) []   link » We aim to develop an AI-guided autonomous materials design approach to discover high-performance organic photovoltaics (OPVs). Autonomous synthesis, automated characterization, and AI-based methods will be integrated into a closed-loop approach to drive molecular discovery guided by target criteria for OPV performance: efficiency and stability. The long-term goal of the project is two-fold: (1)in terms of fundamental science, we aim to fill key knowledge gaps in understanding how molecular structure determines OPV stability and efficiency, and advance the science of closed-loop autonomous discovery by learning how to synergistically integrate AI, automated synthesis, and automated testing. (2)In terms of technology, we aim to meet the “10-10” target (10\% efficiency and 10-year stability for OPV materials) to make OPVs a commercial reality for next-generation energy capture applications and for mitigating climate change. Link » Changhyun Hwang · Seungjoo Yi · David Friday · Nicholas Angello · Tiara Torres-Flores · Nick Jackson · Martin Burke · Charles Schroeder · Ying Diao 🔗 - Human-in-the-Loop Approaches For Task Guidance In Manufacturing Settings (Poster) []   link » We introduce a task guidance framework for manufacturing settings aiming to improve the well-being and productivity of manufacturing workers completing a given task. The assistive technology proposed in this work centers on a dialogue system built upon semantic frame extraction of process specifications detailing a given manufacturing process. The dialogue system interacts with the technician performing the task by capturing their actions and assisting them in performing relevant steps. Specifically, we develop components to parse expert-authored natural language documents called specs and utilize the parse for task guidance and continual learning. While still in the early stages, we believe that an interactive, assistive AI framework similar to the one we are exploring will become an important component of high-volume manufacturing in the future. Such a system could increase the quality and scalability of next-generation materials and materials-related products, such as batteries or fuel cells, produced by automated materials synthesis techniques and analyzed by automated materials characterization techniques. Link » Ramesh Manuvinakurike · Santiago Miret · Richard Beckwith · Saurav Sahay · Giuseppe Raffa 🔗 - Is GPT-3 all you need for machine learning for chemistry? (Poster) []  []   link » Pre-trained large language models (LLMs) are a powerful platform for building custom models for various applications.They have also found success in chemistry, but typically need to be pre-trained on large chemistry datasets such as reaction databases or protein sequences.In this work, we analyze whether one of the largest pre-trained LLMs, GPT-3, can be directly used for chemistry applications by fine-tuning on only a few data points from a chemistry dataset, i.e., without pre-training on a chemistry-specific dataset.We show that GPT-3 can achieve performance competing with baselines on three case studies (polymers, metal-organic frameworks, photoswitches) with representations as simple as the chemical name in both classification and regression settings.Moreover, we demonstrate that GPT-3 can also be fine-tuned for use in inverse design tasks, i.e., to generate a molecule that has properties as specified in a prompt. Link » Kevin Jablonka · Philippe Schwaller · Berend Smit 🔗 - More trustworthy Bayesian optimization of materials properties by adding human into the loop (Poster) []   link » Bayesian optimization (BO) has proven to be effective approach for guiding sample-efficient exploration of materials domains and is increasingly being used in automated materials optimization set-ups. However, when exploring novel materials, sample quality may vary unexpectedly, which can even invalidate the optimization procedure if it remains undetected. This issue limits the use of highly-automated optimization loops, especially in high-dimensional materials spaces with a lot of samples. Sample quality may be hard to define unequivocally for a machine but human scientists are usually good at judging sample quality, at least on a cursory yet often sufficient level. In this work, we demonstrate that humans can be added into the BO loop as experts to comment on the sample quality, which results in more trustworthy BO results. We implemented human-in-the-loop BO via a data fusion approach and applied virtual BO cycles on experimental perovskite film stability data from literature. The human-in-the-loop approach facilitates automated materials design and characterization by reducing the occurrence of invalid optimization results. Link » Armi Tiihonen · Louis Filstroff · Petrus Mikkola · Emma Lehto · Samuel Kaski · Milica Todorović · Patrick Rinke 🔗 - Actively Learning Costly Reward Functions for Reinforcement Learning (Poster) []   link » Transfer of recent advances in deep reinforcement learning to real-worldapplications is hindered by high data demands and thus low efficiency andscalability. Through independent improvements of components such as replaybuffers or more stable learning algorithms, and through massively distributedsystems, training time could be reduced from several days to several hours forstandard benchmark tasks. However, while rewards in simulated environments arewell-defined and easy to compute, reward evaluation becomes the bottleneck inmany real-world environments, e.g. in molecular optimization tasks, wherecomputationally demanding simulations or even experiments are required toevaluate states and to quantify rewards. Therefore, training might becomeprohibitively expensive without an extensive amount of computational resourcesand time. We propose to alleviate this problem by replacing costly ground-truthrewards with rewards modeled by neural networks, counteracting non-stationarityof state and reward distributions during training with an active learning component.We demonstrate that using our proposed ACRL method (actively learning costlyrewards for reinforcement learning), it is possible to train agents in complexreal-world environments orders of magnitudes faster. By enabling the applicationof reinforcement learning methods to new domains, we show that we can findinteresting and non-trivial solutions to real-world optimization problems inchemistry, materials science and engineering. Link » André Eberhard · Houssam Metni · Georg Fahland · Alexander Stroh · Pascal Friederich 🔗 - Deep Reinforcement Learning for Inverse Inorganic Materials Design (Poster) []  []   link » A major obstacle to the realization of novel inorganic materials with desirable properties is the ability to perform efficient optimization across both materials properties and synthesis of those materials. In this work, we propose a reinforcement learning (RL) approach to inverse inorganic materials design, which can identify promising compounds with specified properties and synthesizability constraints. Our model learns chemical guidelines such as charge and electronegativity neutrality while maintaining chemical diversity and uniqueness. We demonstrate a multi-objective RL approach, which can generate novel compounds with targeted materials properties including formation energy and bulk/shear modulus alongside a lower sintering temperature synthesis objectives. Using this approach, the model can predict promising compounds of interest, while informing an optimized chemical design space for inorganic materials discovery. Link » Elton Pan · Christopher Karpovich · Elsa Olivetti 🔗 - MEGAN: Multi Explanation Graph Attention Network (Poster) []  []   link » Besides increasing trust in the human-AI relationship, XAI methods have the potential to promote new scientific insight. Graph neural networks (GNNs) have recently established themselves as a valuable tool in chemistry and material sciences. Various XAI methods have already been applied to gain new understanding of real-world scientific questions in these application domains. To that end we propose MEGAN, a multi-explanation graph attention network. Unlike common post-hoc XAI methods, our model is self-explaining and features multiple explanation channels, which can be chosen independent of the task specifications. We first validate our model on a synthetic graph regression dataset. We then apply our model to the prediction of water solubility for chemical compounds. We find that it learns to produce explanations consistent with human intuition, opening the way to learning from our model in less well-understood tasks. Link » Jonas Teufel · Luca Torresi · Patrick Reiser · Pascal Friederich 🔗 - Transfer Learning Lithium and Electrolyte Potential Energy Surfaces from Pure and Hybrid DFT (Poster) []  []   link » One of the most important problems in rational design of batteries is predicting the properties of the Solid Electrolyte Interphase, which (for a metallic anode) is the part of the battery where metallic and non-metallic components come into contact. However, there is a fundamental problem with predicting the properties of such a mixed material: the two components are best simulated with incompatible levels of density functional theory. Pure functionals perform well for metallic properties, while hybrid or long-range-corrected density functionals perform better for molecular properties and reaction barriers. We demonstrate a simple method to obviate this conflict by training a machine learning potential energy surface using both levels of theory via transfer learning. We further show that the resulting model is more accurate than models trained individually to these levels of theory, allowing more accurate property prediction and potentially faster materials discovery. Link » James Stevenson · Leif Jacobson · Garvit Agarwal · Steven Dajnowicz 🔗 - Multivariate Prediction Intervals for Random Forests (Poster) []  []   link » Accurate uncertainty estimates can significantly improve the performance of iterative design of experiments, as in Sequential and Reinforcement learning. For many such problems in engineering and the physical sciences, the design task depends on multiple correlated model outputs as objectives and/or constraints. To better solve these problems, we propose a recalibrated bootstrap method to generate multivariate prediction intervals for bagged models and show that it is well-calibrated. We apply the recalibrated bootstrap to a simulated sequential learning problem with multiple objectives and show that it leads to a marked decrease in the number of iterations required to find a satisfactory candidate. This indicates that the recalibrated bootstrap could be a valuable tool for practitioners using machine learning to optimize systems with multiple competing targets. Link » Brendan Folie · Maxwell Hutchinson 🔗 - DeepStruc: Towards structure solution from pair distribution function data using deep generative models (Poster) []  []   link » Structure solution of nanostructured materials that have limited long-range order remains a bottleneck in materials development. We present a deep learning algorithm, DeepStruc, that can solve a simple nanoparticle structure directly from a Pair Distribution Function (PDF) obtained from total scattering data by using a conditional variational autoencoder. We first apply DeepStruc to PDFs from seven different structure types of monometallic nanoparticles, and show that structures can be solved from both simulated and experimental PDFs, including PDFs from nanoparticles that are not present in the training distribution. We also apply DeepStruc to a system of hcp, fcc and stacking faulted nanoparticles, where DeepStruc recognizes stacking faulted nanoparticles as an interpolation between hcp and fcc nanoparticles and is able to solve stacking faulted structures from PDFs. Our findings suggests that DeepStruc is a step towards a general approach for structure solution of nanomaterials. Link » Emil Thyge Skaaning Kjær · Andy S. Anker · Marcus Weng · Simon J. L. Billinge · Raghavendra Selvan · Kirsten Jensen 🔗 - A Generalized Framework for Microstructural Optimization using Neural Networks (Poster) []  []   link » Microstructures, i.e., architected materials, are designed today, typically, by maximizing an objective, such as bulk modulus, subject to a volume constraint. However, in many applications, it is often more appropriate to impose constraints on other physical quantities of interest.In this paper, we consider such generalized microstructural optimization problems where any of the microstructural quantities, namely, bulk, shear, Poisson ratio, or volume, can serve as the objective, while the remaining can serve as constraints. In particular, we propose here a neural-network (NN) framework to solve such problems. The framework relies on the classic density formulation of microstructural optimization, but the density field is represented through the NN's weights and biases.The main characteristics of the proposed NN framework are: (1) it supports automatic differentiation, eliminating the need for manual sensitivity derivations, (2) smoothing filters are not required due to implicit filtering, (3) the framework can be easily extended to multiple-materials, and (4) a high-resolution microstructural topology can be recovered through a simple post-processing step. The framework is illustrated through a variety of microstructural optimization problems. Link » Saketh Sridhara · Aaditya Chandrasekhar · Krishnan Suresh 🔗 - Graph Contrastive Learning for Materials (Poster) []  []   link » Recent work has shown the potential of graph neural networks to efficiently predict material properties, enabling high-throughput screening of materials. Training these models, however, often requires large quantities of labelled data, obtained via costly methods such as ab initio calculations or experimental evaluation. By leveraging a series of material-specific transformations, we introduce CrystalCLR, a framework for constrastive learning of representations with crystal graph neural networks. With the addition of a novel loss function, our framework is able to learn representations competitive with engineered fingerprinting methods. We also demonstrate that via model finetuning, contrastive pretraining can improve the performance of graph neural networks for prediction of material properties and significantly outperform traditional ML models that use engineered fingerprints. Lastly, we observe that CrystalCLR produces material representations that form clusters by compound class. Link » Teddy Koker · Keegan Quigley · Will Spaeth · Nathan Frey · Lin Li 🔗 - Group SELFIES: A Robust Fragment-Based Molecular String Representation (Poster) []   link » The design of functional molecules relies on the representation used: a flexible and informative representation can improve downstream generation tasks. String representations such as SMILES and SELFIES serve as the basis for chemical language models, and the robustness of SELFIES makes it naturally suited for molecular optimization with genetic algorithms. But while SMILES and SELFIES are atomic representations, several recent approaches take advantage of the inductive bias of molecular fragments. In this work, we present Group SELFIES, introducing group tokens that represent functional groups or entire substructures while maintaining robustness. Group tokens give control over which structures should be preserved during optimization. Experiments indicate that Group SELFIES improves distribution learning and improves the quality of molecules generated by simply taking random Group SELFIES strings. The code is available at \url{https://anonymous.4open.science/r/group-selfies-4D87/}. Link » Austin Cheng · Andy Cai · Santiago Miret · Gustavo Malkomes · Mariano Phielipp · Alan Aspuru-Guzik 🔗 - Extracting Structural Motifs from Pair Distribution Function Data of Nanostructures using Explainable Machine Learning (Poster) []  []   link » Characterization of material structure with X-ray or neutron scattering using e.g. Pair Distribution Function (PDF) analysis most often rely on refining a structure model against an experimental dataset. However, identifying a suitable model is often a bottleneck. Recently, new automated approaches have made it possible to test thousands of models for each dataset, but these methods are computationally expensive, and analysing the output, i.e., extracting structural information from the resulting fits in a meaningful way is challenging. Our Machine Learning based Motif Extractor (ML-MotEx) trains an ML algorithm on thousands of fits, and uses SHAP (SHapley Additive exPlanation) values to identify which model features are important for the fit quality. We use the method for 4 different chemical systems including disordered nanomaterials and clusters. ML-MotEx opens for a new type of modelling where each feature in a model is assigned an importance value for the fit quality based on explainable ML. Link » Andy S. Anker · Emil Thyge Skaaning Kjær · Mikkel Juelsholt · Troels Christiansen · Susanne Skjærvø · Mads Jørgensen · Innokenty Kantor · Daniel Sørensen · Simon J. L. Billinge · Raghavendra Selvan · Kirsten Jensen 🔗 - Neural Structure Fields with Application to Crystal Structure Auto-Encoders (Poster) []  []   link » Representing crystal structures of materials in a form preferable for neural networks is crucial for enabling machine learning applications involving the estimation of crystal structures. This paper proposes Neural Structure Fields (NeSF) as an accurate and practical approach to representing crystal structures by neural networks. Our crucial idea, inspired by the concepts of both vector fields in physics and implicit neural representations in computer vision, is to consider a crystal structure as a continuous field rather than a discrete set of atoms. Unlike existing grid-based discretized spatial representations, NeSF is free from a trade-off between spatial resolution and computational complexity and can represent complex crystal structures. We demonstrate the expressibility of NeSF successfully in applying it to auto-encoding of crystal structures. Link » Naoya Chiba · Yuta Suzuki · Tatsunori Taniai · Ryo Igarashi · Yoshitaka Ushiku · Kotaro Saito · Kanta Ono 🔗 - Hyperparameter Optimization of Graph Neural Networks for the OpenCatalyst Dataset: A Case Study (Poster) []  []   link » The proliferation of deep learning (DL) techniques in recent years has often resulted in the creation of progressively larger datasets and deep learning architectures. As the expressive power of DL models has grown, so has the compute capacity needed to effectively train the models. One such example is the OpenCatalyst dataset in the emerging field of scientific machine learning, which has elevated the compute requirements needed to effectively train graph neural networks (GNNs) on complex scientific data. The extensive compute complexity involved in training GNNs on the OpenCatalyst dataset makes it very costly to perform hyperparameter optimization (HPO) using traditional methods, such as grid search or even Bayesian optimization-based approaches. Given this challenge, we propose a novel methodology for effective, cost-aware HPO on GNN training on OpenCatalyst that leverages a multi-fidelity approach with experiments on reduced datasets, hyperparameter importance, and computational budget considerations. We show speed ups by over 50 percent when performing hyperparameter optimization of the E(n)-GNN model on the OpenCatalyst dataset. Link » Carmelo Gonzales · Eric Lee · Kin Long Kelvin Lee · Joyce Tang · Santiago Miret 🔗 - AI-assisted chemical reaction impurity prediction and propagation (Poster) []   link » Most chemical reactions result in numerous by-products and side-products, apart from the intended major product. While chemists can predict many of the main process impurities, it remains a challenge to enumerate the possible minor impurities and even more of a challenge to systematically predict and track impurities derived from raw materials or those that have propagated from one synthetic step to the next. In this study, we developed an AI-assisted approach to predict and track impurities across multi-step reactions using the main reactants, and optionally reagents, solvents and impurities in these materials, as input. We demonstrated the utility of this tool for a simple case of synthesis of paracetamol from phenol, and provide a generalized framework that covers most chemical reactions. Our solution can be applied to enable (1) faster elucidation of impurities, (2) automated interpretation of data generated from high-throughput reaction screening, and (3) more thorough raw materials risk assessments, with each of these representing key workflows in small molecule drug substance commercial process development. Link » Somesh Mohapatra · Daniel Griffin 🔗 - Conformer Search Using SE3-Transformers and Imitation Learning (Poster) []  []   link » We introduce a novel approach to conformer search, the discovery of three-dimensional structures for two-dimensional molecular formulas. We focus on organic molecules using deep imitation learning and equivariant graph neural networks, with the prospect of using reinforcement learning algorithms for fine tuning. To that end, we present our interactive environment that describes the molecule in a ridig-rotor approximation and leverage a behavioral cloning torsion policy to autoregressively determine the dihedral angles of the molecule ultimately yielding a three-dimensional molecular structure. For our policy architecture, we leverage an SE(3) equivariant neural network, which enables us to exploit inherent molecular symmetries and to respect the topology of the angle distribution using a Mixture of Projected Normals action distribution. Our preliminary results for a policy trained on a behavioral cloning objective using the QM9 dataset for expert trajectories shows that the policy can accurately predict torsion angles for various molecules. We believe this to be a promising starting point for future work pertaining to performing conformer search using deep reinforcement learning. Link » Luca Thiede · Santiago Miret · Krzysztof Sadowski · Haoping Xu · Mariano Phielipp · Alan Aspuru-Guzik 🔗 - A deep learning and data archaeology approach for mosquito repellent discovery (Poster) []  []   link » Insect-borne diseases kill >0.5 million people annually. Currently available repellents for personal or household protection are limited in their efficacy, applicability, and safety profile. Here, we describe a machine-learning-driven high-throughput method for the discovery of novel repellent molecules. To achieve this, we digitized a large, historic dataset containing ~19,000 mosquito repellency measurements. We then trained a graph neural network (GNN) to map molecular structure and repellency. We applied this model to select 317 candidate molecules to test in parallelizable behavioral assays, quantifying repellency in multiple pest species and in follow-up trials with human volunteers. The GNN approach outperformed a chemoinformatics model and produced a hit rate that increased with training data size, suggesting that both model innovation and novel data collection were integral to predictive accuracy. We identified >10 molecules with repellency similar to or greater than the most widely used repellents. This approach enables computational screening of billions of possible molecules to identify empirically tractable numbers of candidate repellents, leading to accelerated progress towards solving a global health challenge. Link » Jennifer Wei · Marnix Vlot · Benjamin Sanchez-Lengeling · Brian Lee · Luuk Berning · Martijn Vos · Rob Henderson · Wesley Qian · D. Michael Ando · Kurt Groetsch · Richard Gerkin · Alexander Wiltschko · Koen Dechering 🔗

#### Author Information

##### Shyue Ping Ong (University of California-San Diego)

Prof Shyue Ping Ong is a Professor of NanoEngineering at the University of California, San Diego. He obtained his PhD from the Massachusetts Institute of Technology in 2011. He leads the Materials Virtual Lab at UCSD, a dynamic group of materials scientists focusing on the interdisciplinary application of materials science, computer science, and data science to accelerate materials design. He is one of the founding developers of the Materials Project, a DOE-funded initiative to make the computed properties of all known materials publicly available for materials innovation. He is also the founder of Python Materials Genomics (pymatgen), an open-source materials analysis library that is used by hundreds of thousands of users worldwide. At UCSD, Dr Ong has served in the NanoEngineering Department’s Graduate Affairs Committee, Undergraduate Affairs Committee, as well as leads the Alumni Newsletter Committee. Dr Ong is also a recipient of the US Department of Energy Early Career Research Program and the Office of Naval Research Young Investigator Program awards.