Representing crystal structures of materials in a form preferable for neural networks is crucial for enabling machine learning applications involving the estimation of crystal structures. This paper proposes Neural Structure Fields (NeSF) as an accurate and practical approach to representing crystal structures by neural networks. Our crucial idea, inspired by the concepts of both vector fields in physics and implicit neural representations in computer vision, is to consider a crystal structure as a continuous field rather than a discrete set of atoms. Unlike existing grid-based discretized spatial representations, NeSF is free from a trade-off between spatial resolution and computational complexity and can represent complex crystal structures. We demonstrate the expressibility of NeSF successfully in applying it to auto-encoding of crystal structures.