Skip to yearly menu bar Skip to main content


Poster

Optimal Hypothesis Selection in (Almost) Linear Time

Maryam Aliakbarpour · Mark Bun · Adam Smith

West Ballroom A-D #5710
[ ]
[ Paper [ Poster [ OpenReview
Thu 12 Dec 4:30 p.m. PST — 7:30 p.m. PST

Abstract: Hypothesis selection, also known as density estimation, is a fundamental problem in statistics and learning theory. Suppose we are given a sample set from an unknown distribution P and a finite class of candidate distributions (called hypotheses) H\coloneqq{H1,H2,,Hn}. The aim is to design an algorithm that selects a distribution H^ in H that best fits the data. The algorithm's accuracy is measured based on the distance between H^ and P compared to the distance of the closest distribution in H to P (denoted by OPT). Concretely, we aim for H^PTV to be at most αOPT+ϵ for some small ϵ and α. While it is possible to decrease the value of ϵ as the number of samples increases, α is an inherent characteristic of the algorithm. In fact, one cannot hope to achieve α<3 even when there are only two candidate hypotheses, unless the number of samples is proportional to the domain size of P [Bousquet, Kane, Moran '19]. Finding the best α has been one of the main focuses of studies of the problem since early work of [Devroye, Lugosi '01]. Prior to our work, no algorithm was known that achieves α=3 in near-linear time. We provide the first algorithm that operates in almost linear time (O~(n/ϵ3) time) and achieves α=3. This result improves upon a long list of results in hypothesis selection. Previously known algorithms either had worse time complexity, a larger factor α, or extra assumptions about the problem setting.In addition to this algorithm, we provide another (almost) linear-time algorithm with better dependency on the additive accuracy parameter ϵ, albeit with a slightly worse accuracy parameter, α=4.

Chat is not available.