StarCraft Motion: A Dataset for Agent Simulation in Adversarial and Partially Observable Scenarios

Agent simulation aims to model the future behavior of interacting agents, but existing benchmarks have largely focused on structured domains such as autonomous driving. Adversarial and partially observable settings remain less explored, despite their importance for modeling strategic multi-agent behavior. To address this gap, we introduce StarCraft Motion, a large-scale dataset for agent simulation built from human StarCraft~II replays. Our processing pipeline converts raw replays into standardized simulation scenarios by extracting continuous unit trajectories, deriving command-based unit-level intent labels, and aligning game states with player-specific fog-of-war observations. This enables direct study of behavior, intent, and opponent responses under competitive partial observability. As an initial study, we evaluate autoregressive simulation methods on StarCraft Motion and introduce HMART, a hierarchical baseline that extends SMART with learnable player-level query aggregation for efficient global-context modeling in large-agent-count scenarios. Experiments show that global context improves observer-unit simulation and intent prediction, while closed-loop fine-tuning further improves rollout quality. However, accurately capturing intent initiation and transitions remains difficult. We also find that opponent modeling from partial observations is substantially more challenging, and that additional global information can be misleading when it does not match the opponent's perspective. These results highlight StarCraft Motion as a useful dataset for studying scalable agent simulation in adversarial, partially observable environments.