Stochastically constrained adversarial lifelong learning
Date
2026
Authors
Grant-Hagen, Mica
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Lifelong learning is a setting of online learning where there is a sequence of tasks and each task is an online learning problem. The lifelong learning algorithm is split into two sections: the meta-level and the within-task level. In this thesis, we explore an online learning setting with a stochastically-constrained adversarial (SCA) assumption. The SCA assumption is that there is not a singular representation that works the best for each of the tasks in terms of minimizing expected losses, but that there is a representation that will become better than all other representations on average over all tasks after enough tasks have been complete. We show that in the single task setting, Decreasing Hedge with perturbed losses achieves best-of-both-worlds regret bounds under the SCA assumption. In the lifelong learning setting, we propose an algorithm with Decreasing Hedge in the meta-level and Squint in the within-task level. Under the SCA assumption in the meta-level and the Bernstein condition in the within-task level, this lifelong learning algorithm achieves best-of-both-worlds regret bounds.
Description
Keywords
Lifelong learning, Online Learning Theory, Machine learning theory