Poster
Nearly Optimal Bounds for Cyclic Forgetting
William Swartworth · Deanna Needell · Rachel Ward · Mark Kong · Halyun Jeong
Great Hall & Hall B1+B2 (level 1) #1807
Abstract:
We provide theoretical bounds on the forgetting quantity in the continual learning setting for linear tasks, where each round of learning corresponds to projecting onto a linear subspace. For a cyclic task ordering on tasks repeated times each, we prove the best known upper bound of on the forgetting. Notably, our bound holds uniformly over all choices of tasks and is independent of the ambient dimension. Our main technical contribution is a characterization of the union of all numerical ranges of products of (real or complex) projections as a sinusoidal spiral, which may be of independent interest.
Chat is not available.