From Simulator Labels to Real Markets: Crumble-Aware Reinforcement Learning for Trade Execution

Displayed liquidity in an electronic limit order book can disappear within moments when top-of-book quotes are withdrawn faster than they are replenished, a pattern known as crumbling quotes. In real markets such deterioration can be driven by strategic market-maker behavior responding to order-flow toxicity, so a one-sided withdrawal can dislocate the touch and precede a price move. Because real markets provide no ground-truth labels for mechanically driven crumbling, we generate a controlled analogue in an ABIDES multi-agent simulation, inducing one-sided quote starvation by switching a market-maker agent’s quoting regime under our control. We use these labels to train a transformer that scores, from recent order-flow context, whether observed liquidity deterioration matches this controlled mechanical pattern. Although trained entirely in simulation, the detector’s score transfers to real equity data, where it is associated with the direction and magnitude of a transient price move. We then provide this signal to a deep Q-network execution agent that learns when to cross the spread, post passively, or wait. Across the tested trading days and equities, the agent lowers execution cost and improves execution gain/loss relative to an aggressive market-order strategy and reinforcement-learning baselines. These results show that simulation-based labeling can recover a transferable liquidity-depletion signal with practical value for trade execution.