Thematic Einstein Semester on Mathematical Optimizationfor Machine Learning

Mathematical optimization often focuses on accuracy, computational efficiency, and robustness while
machine learning (ML) aims at achieving effective results on real datasets, in particular aiming for generalization, robustness, and resilience (to e.g., perturbations of the inputs). Up to now, both research
areas are only loosely intertwined with optimization being the ‘tool’ to execute the learning task from the
ML perspective, and ML being just ‘yet another application’ from the optimization perspective. This is
illustrated by the fact that certain variants of stochastic gradient descent (e.g., out-of-the-box Adam) are
still the method of choice in machine learning despite the fact that the application of this method is not
justified by theory for a large number of applications from machine learning. On the other hand, mathematical optimization often studies approximation properties for machine learning tasks but does so far not
develop optimization approaches targeting the needs of machine learning (e.g., aiming for generalization
or sparsity). Hence, there remains a dearth of work to advance optimization techniques to such an extent
that machine learning problems can be handled with the required efficiency. Connecting mathematical
optimization perspectives with machine learning approaches will help to generate ideas for new approaches
or improvements in existing approaches. For instance, the problem of predicting cluster membership in unsupervised learning may be greatly aided by an optimization perspective; after all, as with many other data
science approaches, the problem can be viewed as maximizing a (complicated) function subject to some
constraints. Complementing this, machine learning heavily influences downstream decision making and
more generally decision support systems in many real-world applications across industries. One prominent
set of examples of such approaches that made headway is motivated by tasks originating from the transition
to renewable energy sources including energy generation, storage, transmission and delivery, and trading
fitting perfectly to the MATH+ slogan “Transforming the World through Mathematics”, AA3 “Networks”
as well as AA4 “Energy and Markets”. As such the proposed TES naturally connects to MATH+ and
sets out to explore new perspectives that might lead to new developments that will likely also impact
Math+ research beyond the TES. Moreover, the TES will also help to maintain the current momentum in
mathematical optimization and machine learning for cross-disciplinary research between the two fields