Sparse Interpretable Deep Learning with LIES Networks for Symbolic Regression

Symbolic regression (SR) aims to recover interpretable closed-form expressions from data. Existing methods often rely on population-based search and autoregressive modeling which struggle with scalability and reliability. We introduce LIES (Logarithm, Identity, Exponential, Sine), a fixed neural network architecture with interpretable primitive activations optimized to model symbolic expressions. We develop a framework to extract compact formulae from LIES networks by training with an appropriate oversampling strategy and a tailored loss function to promote sparsity and to prevent gradient instability. After training, it applies additional pruning strategies to further simplify the learned expressions into compact formulae. Our experiments on SR benchmarks show that the LIES framework consistently produces sparse and accurate symbolic formulae outperforming all baselines. We also demonstrate the importance of each design component through ablation studies.