Skip to yearly menu bar Skip to main content


Poster

Nearly Optimal Bounds for Cyclic Forgetting

William Swartworth · Deanna Needell · Rachel Ward · Mark Kong · Halyun Jeong

Great Hall & Hall B1+B2 (level 1) #1807

Abstract: We provide theoretical bounds on the forgetting quantity in the continual learning setting for linear tasks, where each round of learning corresponds to projecting onto a linear subspace. For a cyclic task ordering on T tasks repeated m times each, we prove the best known upper bound of O(T2/m) on the forgetting. Notably, our bound holds uniformly over all choices of tasks and is independent of the ambient dimension. Our main technical contribution is a characterization of the union of all numerical ranges of products of T (real or complex) projections as a sinusoidal spiral, which may be of independent interest.

Chat is not available.