PRISM:Disentangling Preference Distributions for Generative Ranking

Preference ranking modeling is important for selecting and adapting language-model responses to human preferences. Compared with deterministic ranking methods, generative preference ranking models the prompt-response data distribution, providing richer uncertainty characterization and sample diversity for learning a more expressive ranking boundary. Recent work improves modeling efficiency by shifting generative preference ranking from textual space to embedding space, but without explicit preference-conditioned separation, the learned embedding distribution may still entangle preferred and rejected responses, producing ambiguous synthetic samples that weaken the ranking boundary. To address this issue, we propose PRISM, a Preference Ranking framework through dISentangled embedding Modeling. PRISM formulates embedding-level generative preference ranking with a unified class-conditional ELBO and decomposes this optimization objective to expose the encoded latent entanglement between preferred and rejected embeddings. This derivation motivates two practical preference-aware generation variants: PRISMMI, involving a deep class-separation objective, and PRISMMMD, catering a probabilistic aggregate-matching objective. Both variants learn a preference-disentangled embedding data distribution and synthesize data pairs that preserve ranking semantics for the generative ranking boundary. Experiments show that PRISM improves preference ranking performance and produces useful generated embeddings for downstream response selection. The anonymous code repository is available at:https://anonymous.4open.science/r/PRISM-915C/README.md.