Probabilistic modelling provides a mathematical framework for understanding what learning is, and has therefore emerged as one of the principal approaches for designing computer algorithms that learn from data acquired through experience. I will review the foundations of this field, from basics to Bayesian nonparametric models and scalable inference. I will then highlight some current areas of research at the frontiers of machine learning, leading up to topics such as probabilistic programming, Bayesian optimisation, the rational allocation of computational resources, and the Automatic Statistician.

Arthur Winfree was one of the pioneers who postulated that several diseases are actually disorders of dynamics of biological systems. Following this path, many now believe that psychiatric diseases are disorders of brain dynamics. Combination of noninvasive brain measurement techniques, brain decoding and neurofeedback, and machine learning algorithms opened up a revolutionary pathway to quantitative diagnosis and therapy of neuropsychiatric disorders.

Recent progress in machine applications of deep neural networks have highlighted the need for a theoretical understanding of the capacity and limitations of these architectures. I will review our understanding of sensory processing in such architectures in the context of the hierarchies of processing stages observed in many brain systems. I will also address the possible roles of recurrent and top - down connections, which are prominent features of brain information processing circuits.

Motivated by machine learning problems over large data sets and distributed optimization over networks, we consider the problem of minimizing the sum of a large number of convex component functions. We study incremental gradient methods for solving such problems, which use information about a single component function at each iteration. We provide new convergence rate results under some assumptions. We also consider incremental aggregated gradient methods, which compute a single component function gradient at each iteration while using outdated gradients of all component functions to approximate the entire global cost function, and provide new linear rate results.

This is joint work with Mert Gurbuzbalaban and Pablo Parrilo.

In the talk, I will introduce a model of learning with Intelligent Teacher. In this model, Intelligent Teacher supplies (some) training examples $\mathscr{(x_i, y_i), i=1, \dots , l, x_i \in X,y_i \in \{-1,1\}}$ with additional (privileged) information) $\mathscr{x_i^* \in X^*}$ forming training triplets $\mathscr (x_i,x_i^*, y_i), i, \dots , l$. Privileged information is available only for training examples and $not\, available\, for\, text\, examples$. Using privileged information it is required to find a better training processes (that use less examples or more accurate with the same number of examples) than the classical ones. In this lecture, I will present two additional mechanisms that exist in learning with Intelligent Teacher * The mechanism to control Student's concept of examples similarity and * The mechanism to transfer knowledge that can be obtained in space of privileged information to the desired space of decision rules. Privileged information exists for many inference problem and Student-Teacher interaction can be considered as the basic element of intelligent behavior.

Talk Slides »

In this talk I will present new inference tools for adaptive statistical procedures. These tools provide p-values and confidence intervals that have correct "post-selection" properties: they account for the selection that has already been carried out on the same data. I discuss application of these ideas to a wide variety of problems including Forward Stepwise Regression, Lasso, PCA, and graphical models. I will also discuss computational issues and software for implementation of these ideas.

This talk represents work (some joint) with many people including Jonathan Taylor, Richard Lockhart, Ryan Tibshirani, Will Fithian, Jason Lee, Dennis Sun, Yuekai Sun and Yunjin Choi.