Attention Redistribution During Event Segmentation In Large Language Model

Human beings perceive a continuous string of experiences by segregating the experience into discrete events. Recently, it has been proven that a large language model can segregate events similarly to humans, even though the model is not specifically trained to do so. In this research, we used naturalistic stimuli like stories to explore the underlying changes in the attention mechanisms when large language model performs event segmentation. We discovered a redistribution of attention outputs toward words that play different roles in structuring an event. We found that the model enhances attention directed toward words indicative of potential changes in elements like time, space, objects, and goals in a continuous narrative. The model also reduces attention directed toward other kinds of words not indicative of such change. Our results provide better insights into the underlying processes of the high-level cognitive features in large language models and in the human brain.