Projection Methods: Algorithmic Structures, Bregman Projections, and Acceleration Techniques

Recognizing that the Perceptron algorithm is a projection method inspires our interest in the class of projection methods. Are there other methods in that class which can be interpreted as, or modified into, machine learning algorithms? What else can methods from this class "do" for us? How do projection methods work? Can they employ different kinds of projections, such as Bregman projections? What are the algorithmic structures available in the class of projection methods? what can be said about convergence properties and/or experimental initial behavior patterns? When, why and how are projection methods preferable over other methods? How can they be accelerated? We will touch upon these questions and explain why this class of methods has witnessed great progress in recent years. Some significant real-world applications benefit from the use of projection methods. We will briefly describe how the fully-discretized model in image reconstruction from projections and the inverse problem of intensity-modulated radiation therapy (IMRT) lend themselves to such methods. Finally, recent work in machine learning is being cross-fertilized by various projection methods.