PhD Dissertation Proposal by Doyen SAHOO

Recent years have witnessed the success of two broad categories of machine learning algorithms: (i) Online Learning; and (ii) Learning with nonlinear models. Typical machine learning algorithms assume that the entire data is available prior to the training task. This is often not the case in the real world, where data arrives sequentially in a stream, or is too large to be stored in memory. To address these challenges, Online Learning techniques evolved as a promising solution to having highly scalable and efficient learning methods which could learn from data arriving sequentially. Next, as the real world data exhibited complex nonlinear patterns, it warranted the need for development of learning techniques that could search complex hypotheses space. Among the most notable successful methods for learning nonlinear models are kernel methods and deep neural networks. While these models enable searching complex hypothesis to learn models with a better performance, they are mostly designed for the batch setting which affects their scalability, and they also suffer from the difficulty in selecting the right hypothesis search space (e.g. which kernel to use, what architecture of neural network to use, etc.). In this work we study the intersection of both these fields, and design novel algorithms that combine the merits of both online learning and nonlinear models by proposing methods that can learn nonlinear models in an online setting. Specifically, we investigate Online Learning with Multiple Kernels. We develop online multiple kernel regression for regression and time series tasks; temporal kernel descriptors to capture temporal properties of data; and cost-sensitive online multiple kernel algorithms for imbalanced data streams. We also suggest directions for developing Online Deep Learning.