Escaping the Obvious: Reinforcement Learning for Conceptually Diverse Generation

Large language models often converge on a narrow set of high-probability ideas when asked for novel associations, analogies, concepts, or creative outputs. This constrains the users' exploratory agency by repeatedly generating output from familiar conceptual regions. We investigate whether a simple reinforcement-learning objective can counter this tendency by directly rewarding further conceptual exploration. We post-train Qwen3-14B using Group Relative Policy Optimization (GRPO), rewarding outputs that connect the prompted topic to a semantically distant secondary topic. We evaluate the method on three creative writing tasks: analogy, poetry, and flash fiction. These tasks were chosen because they impose different structural constraints while making conceptual movement observable. We evaluate within-prompt and global diversity, semantic geometry, baseline-calibrated occupancy, and coherence. Results show that GRPO training increases the semantic distance between concepts across all three tasks and shifts the mean trajectory of generated outputs toward less occupied regions of the embedding space. The secondary topics of generated output move from literal associations or near-neighbor clusters (for example, volcano–eruption or candlelight–shadow) toward more semantically distant but usable topic associations (volcano–argument or candlelight–intruder). Our method helps language models to explore more diverse and semantically distant topics, which is useful for tasks that benefit from novelty and conceptual breadth, such as creative writing, ideation, design, brainstorming, and hypothesis generation.