Intelligent agents must be able to learn by interacting with their environment and to adapt to changes. Continual reinforcement learning provides a natural way to model this process. In this talk, I will discuss for tackling this problem by constructing abstractions, such as intents, options, affordances and partial models that allow an agent to generalize its knowledge quickly to new circumstances.