Numerical Mathematics Challenges in Machine Learning

Most machine learning (ML) methods are based on numerical mathematics (NM)
concepts, from differential equation solvers over dense matrix factorizations
to iterative linear system and eigen-solvers. For problems of moderate size,
NM routines can be invoked in a black-box fashion. However, for a growing
number of real-world ML applications, this separation is insufficient and
turns out to be a limit on further progress.\par

The increasing complexity of real-world ML problems must be met with layered
approaches, where algorithms are long-running and reliable components rather
than stand-alone tools tuned individually to each task at hand. Constructing
and justifying dependable reductions requires at least some awareness about NM
issues. With more and more basic learning problems being solved sufficiently
well on the level of prototypes, to advance towards real-world practice the
following key properties must be ensured: scalability, reliability, and
numerical robustness. \par

By inviting numerical mathematics researchers with interest in both numerical
methodology and real problems in applications close to machine learning, we
will probe realistic routes out of the prototyping sandbox. Our aim is to
strengthen dialog between NM, signal processing, and ML. Speakers are briefed
to provide specific high-level examples of interest to ML and to point out
accessible software. We will initiate discussions about how to best bridge gaps
between ML requirements and NM interfaces and terminology. \par

The workshop will reinforce the community's awakening attention towards
critical issues of numerical scalability and robustness in algorithm design
and implementation. Further progress on most real-world ML problems is
conditional on good numerical practices, understanding basic robustness and
reliability issues, and a wider, more informed integration of good numerical
software. As most real-world applications come with reliability and scalability
requirements that are by and large ignored by most current ML methodology, the
impact of pointing out tractable ways for improvement is substantial.

\par\noindent Target audience: \par

Our workshop is targeted towards practitioners from NIPS, but is of interest
to numerical linear algebra researchers as well.