Position: Trust Hinders Bidirectional Human-AI Alignment in Decision-Making

Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users exhibit appropriate reliance on them. We argue that this approach, while useful for decision problems that are well defined and scoped, overly restricts what it means to make a decision with AI support. To overcome this limitation we develop a three-part argument. First, trusting AI is only one of several possible strategies for managing uncertainty. Second, trust calibration has been misappropriated for AI-modelled “large-world” decision problems, i.e., ill-defined situations where goals and states are imperfectly known or unknowable. Third, in such settings human–AI alignment requires addressing higher-order decision processes. We conclude by reflecting on implications for the design of process-oriented human-AI coupled systems that jointly identify, contest and negotiate uncertainty throughout the decision-making process.