Learning Canonical Cellular Environments from Spatial Transcriptomic Data via Optimal Transport

Cellular environments, or niches, are complex biological systems featuring diverse cell types co-localized and interacting with each other. Because these environments orchestrate important functions such as the immune response and stem cell differentiation, it is imperative that we study cells in their spatial context. Although spatial transcriptomic technologies such as MERFISH measure location and gene expressions at the resolution of individual cells, there is a lack of specialized methods to reason about the cellular environments in these datasets. We propose a framework to analyze cellular environments in spatial transcriptomic data, featuring principled methods to represent environments, measure their similarities, and cluster them in order to learn a set of representative, canonical environments. We apply our method on mouse primary motor cortex assayed with MERFISH to learn canonical environments which resemble environments in distinct cortex layers, capture the diversity of cell types present in those environments, and reveal gene expression variation across cells of the same cell type within each layer.