Safe-PG-LQR: Provably Safe, Convergent and Optimal Model-Free Linear Quadratic Regulator with Hard Constraints

The linear quadratic regulator (LQR) is a canonical optimal control problem where recent work has established global convergence guarantees for model-free policy gradient methods. However, designing model-free algorithms that enforce hard constraints throughout learning remains an open challenge. Existing approaches are either partially model-based, enforce soft constraints through Lagrangian relaxations, or project pretrained policies onto the constraint set, sacrificing hard constraint satisfaction or convergence guarantees of the underlying algorithms. In this work, we develop principled model-free policy gradient methods for LQR problems with hard constraints. Our key idea is to parameterize the policy class using a truncated stochastic policy so that hard constraints are satisfied by construction, then perform policy optimization entirely within this class. We prove that the resulting LQR objective over the proposed truncated policy class satisfies a weak gradient-dominance property in both the undiscounted and discounted settings. We leverage this property to develop model-free policy-gradient algorithms that maintain zero violation of the encoded hard constraints at each iteration, and converge to an approximately globally optimal policy within the proposed safe policy class.