Spotlight Poster
Capability v.s. Behavior in Compositional Generalization: A Learning Dynamics Perspective
Core Francisco Park · Maya Okawa · Andrew Lee · Ekdeep S Lubana · Hidenori Tanaka
East Exhibit Hall A-C #2911
Studying the learning dynamics of capability (the underlying ability) and behavior (the actual execution of the capability) is critical to understanding generative models. However, studying these notions in large-scale models is challenging due to their complexity. We bridge this gap by simplifying the problem space with synthetic toy data. In this work, we introduce a framework called concept space to study the dynamics of a model's generalization behavior. We introduce the notion of a "concept signal" that governs the rate of concept learning. Using a synthetic dataset of 2D objects with different shapes, sizes, and colors, we study the generalization dynamics of the model in concept space and hypothesize: "Concept learning is a well-controlled phase transition in model capability, but the observed behavior resulting from this transition can be arbitrarily delayed, e.g., by the level of the concept signal." We test our hypothesis with several mechanistic interpretability tools, demonstrating a clear separation of model capability and behavior. Overall, this work defines a framework for understanding and taming the development of capability and behavior.
Live content is unavailable. Log in and register to view live content