ReSHAPE: User-Guided Revision of Planning Specifications

Planning benchmarks commonly treat a user's initial request as a fixed set of constraints and evaluate whether an agent can produce a plan that satisfies them. However, an initial specification may admit acceptable plans while still excluding other acceptable combinations, particularly when the user permits trade-offs across different components of a complete plan. We introduce ReSHAPE, a task that evaluates whether an agent can use interaction to revise such a specification so that the plans it admits better match those the user would accept. We also introduce ShapeForge, a domain-agnostic procedure for augmenting existing planning tasks with stable user profiles, option-unaware initial specifications, simulated feedback, and exact evaluation over the legal plan space. Using ShapeForge, we construct ReSHAPE Travel, a benchmark containing 100 travel-planning instances assigned to 20 users. We evaluate ReSHAPE Travel in two complementary settings: different models under the same interaction protocol, and different interaction protocols under the same model. Results show that current models can substantially improve the initial specification under plan-grounded interaction, whereas Ask-before-Plan and Adaptive Planning are markedly less effective. Further analysis shows that preserving observed feedback does not necessarily yield a specification that generalizes across the full plan space. These results establish interactive specification revision as a distinct challenge from producing a single satisfactory plan.