Infinite-Dimensional HiPPO Provides an Explicit Formula for LSSLs

Recently, models for sequence processing, such as large-scale language models, have become increasingly important. Among these, the Linear State Space Layer (LSSL) has been proposed as a fast sequence-processing model. It is also known that using the HiPPO matrix in these LSSLs improves their performance. In this paper, we extend the HiPPO matrix to an operator on function spaces. Furthermore, we show that the resulting infinite-dimensional LSSL with HiPPO admits an explicit solution.