A rose by any other representation: some questions on the relationship between computation and representation

This workshop focuses on representations, but ultimately, we are interested in these representations primarily for their contribution to a system's computations — either to understand how it works or to improve it. In this talk, I will point out some of the complexities in the relationship between representation and computation. First, certain features may be over-represented or under-represented relative to the role they play in the system's computations, due to learning dynamics or other extrinsic factors. Second, representations and computations may shift under different data distributions. Third, the very notion of a stable representation may be misleading for natural systems. I will suggest that these issues both make the empirical results on representational alignment more surprising and interesting, yet may indicate potential limitations and areas for future investigation.