Persona Vectors: Monitoring and Controlling Character Traits in Language Models

Large language models interact with users through a simulated "Assistant" persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model's activation space - persona vectors - underlying several traits, such as evil, sycophancy, and propensity to hallucinate. We confirm that these vectors can be used to monitor fluctuations in the Assistant's personality at deployment time. We then apply persona vectors to predict and control personality shifts that occur during training. We find that both intended and unintended personality changes after finetuning are strongly correlated with shifts along the relevant persona vectors. A central contribution of our work is preventative steering, a novel training-time intervention that actively suppresses the acquisition of unwanted traits. By steering activations during the fine-tuning process itself, this method “immunizes” the model against negative personality shifts.Moreover, persona vectors can be used to flag training data that will produce undesirable personality changes, both at the dataset level and the individual sample level. Our method for extracting persona vectors is automated and can be applied to any personality trait of interest, given only a natural-language description.