Unsupervised convolutional neural networks-based 3D reconstruction from 2D medical images guided by generative models

Patient-specific features used for surgical planning and custom implant design may require 3D reconstruction of the shape and pose of organs of interest from medical images. Such a task is often accomplished through a challenging problem of fitting a parametric model to the 3D geometry using energy optimization. In this work, we propose a novel generative model-based deep convolutional autoencoder to reconstruct multiple organs in 3D from a 2D image. To this end, we combine a convolutional encoder network with a parametric geometric model that serves as a decoder. The resulting reconstructions compare favorably with current state-of-the-art approaches in terms of accuracy.