Online Multi-Task Learning with Long-Term Memory
Military / Coalition Issue
In military operations we may have a collection of mobile devices that are receiving data from their surroundings, and we may wish to perform some online machine learning task with their collected data. We assume that the machine learning task would be aided by the knowledge of the “environment” that a particular mobile device is currently in. Due to the mobile devices being “mobile” the devices themselves will move between different environments over time. The devices do not necessarily have to move together and hence different devices will be in different environments at different times. We assume we have no a-priori knowledge of the different environments themselves, or when a particular device changes environment, and hence this work is about how to learn these things online.
Core idea and key achievements
We consider two types of online machine learning task which are fundamental in the literature:
- Prediction with expert advice. We assume we have a finite set of algorithms (“experts”). At any point in time these algorithms observe some data (e.g. the current instance of the device’s video feed) and issue a prediction (e.g., whether a threat is detected). The algorithms themselves have different performances in different environments.
- Linear interpolation with a Reproducing Kernel Hilbert Space (RKHS). We have a (potentially infinite) set of functions that map possible data observations to real numbers. A function is consistent with the observed data if it maps each piece of observed data to 0 or 1 (its prediction about that datum). The functions form an RKHS which applies a “complexity” to each function. Functions that are less complex and more natural are easier to learn. The functions vary in predictive performance over the different environments.
In both tasks we learn if the prediction is correct immediately after it is made. We adapt both these learning tasks to our model and for them develop two very different algorithms. This work was done over two papers, both published at NeurIPS 2020: the paper entitled “online matrix completion with side information” developed the mathematical tools needed for the RKHS algorithm, whilst the paper entitled “online multitask learning with long-term memory” introduced both the algorithms themselves.
Implications for Defence
This project develops techniques for performing machine learning on mobile devices which move between different environments.
Readiness & alternative Defence uses
The paper entitled “online multitask learning with long-term memory” contains the pseudo-code for both algorithms.
Resources and references
- Herbster, Mark, Stephen Pasteris, and Lisa Tse. “Online Multitask Learning with Long-Term Memory.” arXiv preprint arXiv:2008.07055 (2020).NeurIPS 2020
- Herbster, Mark, Stephen Pasteris, and Lisa Tse. “Online Matrix Completion with Side Information.” Advances in Neural Information Processing Systems 33 (2020).