Online Inference of Structure Factor Amplitudes for Serial X-ray Crystallography

Advances in X-ray techniques at Free Electron Laser and synchrotrons now enable the collection of diffraction snapshots from millions of micro crystals. These are often paired with physical or chemical perturbations to obtain movies of the response of proteins to chemical and physical stimuli. Analysis of these data requires scalable algorithms. Distributed computing is one way to accomplish this as national labs may provide the necessary compute resources. However, a more accessible approach would be to construct algorithms which can operate on small batches of data on a single computer. The extreme case, an online algorithm, learns to process data by looking at one example at a time. Here we describe the successful implementation of one such algorithm for scaling and merging reflection intensities. The algorithm uses deep learning to scale reflection intensities while encouraging the merged structure factor estimates to follow a crystallographic prior distribution. The model is trained by gradient descent on a Bayesian objective function. We demonstrate that the model can estimate productive global parameter updates from single images. This approach has modest hardware requirements, can adapt on the fly as new data are acquired, and has the potential for transfer learning between data sets. The algorithm can be the heart of a flexible, scalable infrastructure that powers the next generation of diffraction experiments.