Timezone: »

Random Path Selection for Continual Learning
Jathushan Rajasegaran · Munawar Hayat · Salman H Khan · Fahad Shahbaz Khan · Ling Shao

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #25

Incremental life-long learning is a main challenge towards the long-standing goal of Artificial General Intelligence. In real-life settings, learning tasks arrive in a sequence and machine learning models must continually learn to increment already acquired knowledge. The existing incremental learning approaches fall well below the state-of-the-art cumulative models that use all training classes at once. In this paper, we propose a random path selection algorithm, called RPS-Net, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse. Our approach avoids the overhead introduced by computationally expensive evolutionary and reinforcement learning based path selection strategies while achieving considerable performance gains. As an added novelty, the proposed model integrates knowledge distillation and retrospection along with the path selection strategy to overcome catastrophic forgetting. In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity. Through extensive experiments, we demonstrate that the proposed method surpasses the state-of-the-art performance on incremental learning and by utilizing parallel computation this method can run in constant time with nearly the same efficiency as a conventional deep convolutional neural network.

Author Information

Jathushan Rajasegaran (Inception Institute of Artificial Intelligence)

My research interests lie in the general area of deep learning, particularly in deep neural architecture for image classification. I am also interested in statistical machine learning, especially in spectral learning of latent variable models, Life long learning, activity recognition and model compression

Munawar Hayat (IIAI)
Salman H Khan (Inception Institute of Artificial Intelligence)
Fahad Shahbaz Khan (Inception Institute of Artificial Intelligence)
Ling Shao (Inception Institute of Artificial Intelligence)

More from the Same Authors