Predicting Symbolic Propositions in World-Model Latents for Temporally Structured Tasks

Many reinforcement learning tasks require objectives with temporal structure beyond a scalar reward. Reward machines represent such objectives by tracking task progress through symbolic propositions, but typically assume a labelling function mapping observations to propositions. We study the setting where the reward machine is known but this grounding is unavailable at deployment. We propose a world-model-based approach that learns to predict propositions from latent states using privileged labels during training. Building on DreamerV3, these predictions are integrated into imagined rollouts through the given reward machine, with its state conditioning policy learning. This encourages the world-model latent representation to capture task-relevant symbolic information while requiring only raw observations at deployment. Experiments on temporally structured navigation tasks show substantial improvements over generic world-model and privileged-information baselines.