TernaryLM: Memory-Efficient Language Modeling via Native 1.5-Bit Quantization with Adaptive Layer-wise Scaling

Large language models demand substantial compute, limiting deployment on edge and resource-constrained hardware. We present TernaryLM, a 132M-parameter decoder-only transformer trained natively with ternary weights $\{-1, 0, +1\}$ ($\log_2 3 \approx 1.58$ bits/weight) via a straight-through estimator and learnable per-layer scaling, rather than quantizing a pretrained full-precision model post hoc. Trained on 12.9M TinyStories tokens on a single NVIDIA L40S GPU in 1.3 GPU-hours, TernaryLM reaches a validation perplexity of 5.52. An architecturally identical FP32 twin trained with the same data and schedule fits the training set more closely (train perplexity 2.43 vs. 3.35) but overfits, ending at 5.97 and never dropping below 5.58 at any epoch: under identical settings, the ternary constraint narrows the train/validation gap ($1.65\times$ vs. $2.46\times$) and improves held-out perplexity, consistent with regularization rather than underfitting (single seed). Exported with int8 ternary codes and one FP32 scale per projection, TernaryLM needs $1.93\times$ less storage (272.6 MB vs. 527.4 MB; $2.58\times$ with 5-per-byte packing) and $5.2\times$ less peak inference memory than FP32 inference, at a $1.37\times$ latency cost on standard kernels. Under a protocol-matched frozen-backbone probe, TernaryLM matches FP32 GLUE transfer to within 0.5 points, with both limited by the narrow pretraining corpus. Per-layer zero-state occupancy is near-uniform (39-41%) and is almost perfectly explained by weight-distribution shape ($r = 0.97$ with kurtosis), while latent weights progressively concentrate at the quantization threshold. Every result comes from logged single-GPU runs, showing that controlled low-bit language-model research is within reach of modest compute budgets. Code, configurations and training logs are available at https://github.com/1nisharg/TernaryLM-NeurIPS.