Timezone: »
This paper shows how metric learning can be used with Nadaraya-Watson (NW) kernel regression. Compared with standard approaches, such as bandwidth selection, we show how metric learning can significantly reduce the mean square error (MSE) in kernel regression, particularly for high-dimensional data. We propose a method for efficiently learning a good metric function based upon analyzing the performance of the NW estimator for Gaussian-distributed data. A key feature of our approach is that the NW estimator with a learned metric uses information from both the global and local structure of the training data. Theoretical and empirical results confirm that the learned metric can considerably reduce the bias and MSE for kernel regression even when the data are not confined to Gaussian.
Author Information
Yung-Kyun Noh (Seoul National University)
Masashi Sugiyama (RIKEN / University of Tokyo)
Kee-Eung Kim (KAIST)
Frank Park (Seoul National University)
Daniel Lee (Cornell Tech)
More from the Same Authors
-
2020 Poster: Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning »
Yu Yao · Tongliang Liu · Bo Han · Mingming Gong · Jiankang Deng · Gang Niu · Masashi Sugiyama -
2020 Poster: Part-dependent Label Noise: Towards Instance-dependent Label Noise »
Xiaobo Xia · Tongliang Liu · Bo Han · Nannan Wang · Mingming Gong · Haifeng Liu · Gang Niu · Dacheng Tao · Masashi Sugiyama -
2020 Spotlight: Part-dependent Label Noise: Towards Instance-dependent Label Noise »
Xiaobo Xia · Tongliang Liu · Bo Han · Nannan Wang · Mingming Gong · Haifeng Liu · Gang Niu · Dacheng Tao · Masashi Sugiyama -
2020 Poster: Rethinking Importance Weighting for Deep Learning under Distribution Shift »
Tongtong Fang · Nan Lu · Gang Niu · Masashi Sugiyama -
2020 Poster: Learning from Aggregate Observations »
Yivan Zhang · Nontawat Charoenphakdee · Zhenguo Wu · Masashi Sugiyama -
2020 Poster: Analysis and Design of Thompson Sampling for Stochastic Partial Monitoring »
Taira Tsuchiya · Junya Honda · Masashi Sugiyama -
2020 Spotlight: Rethinking Importance Weighting for Deep Learning under Distribution Shift »
Tongtong Fang · Nan Lu · Gang Niu · Masashi Sugiyama -
2020 Poster: Provably Consistent Partial-Label Learning »
Lei Feng · Jiaqi Lv · Bo Han · Miao Xu · Gang Niu · Xin Geng · Bo An · Masashi Sugiyama -
2020 Poster: Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators »
Takeshi Teshima · Isao Ishikawa · Koichi Tojo · Kenta Oono · Masahiro Ikeda · Masashi Sugiyama -
2020 Oral: Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators »
Takeshi Teshima · Isao Ishikawa · Koichi Tojo · Kenta Oono · Masahiro Ikeda · Masashi Sugiyama -
2019 Poster: Uncoupled Regression from Pairwise Comparison Data »
Liyuan Xu · Junya Honda · Gang Niu · Masashi Sugiyama -
2019 Poster: Are Anchor Points Really Indispensable in Label-Noise Learning? »
Xiaobo Xia · Tongliang Liu · Nannan Wang · Bo Han · Chen Gong · Gang Niu · Masashi Sugiyama -
2019 Poster: On the Calibration of Multiclass Classification with Rejection »
Chenri Ni · Nontawat Charoenphakdee · Junya Honda · Masashi Sugiyama -
2018 Poster: Binary Classification from Positive-Confidence Data »
Takashi Ishida · Gang Niu · Masashi Sugiyama -
2018 Spotlight: Binary Classification from Positive-Confidence Data »
Takashi Ishida · Gang Niu · Masashi Sugiyama -
2018 Poster: Uplift Modeling from Separate Labels »
Ikko Yamane · Florian Yger · Jamal Atif · Masashi Sugiyama -
2018 Poster: Continuous-time Value Function Approximation in Reproducing Kernel Hilbert Spaces »
Motoya Ohnishi · Masahiro Yukawa · Mikael Johansson · Masashi Sugiyama -
2018 Poster: A Bayesian Approach to Generative Adversarial Imitation Learning »
Wonseok Jeon · Seokin Seo · Kee-Eung Kim -
2018 Spotlight: A Bayesian Approach to Generative Adversarial Imitation Learning »
Wonseok Jeon · Seokin Seo · Kee-Eung Kim -
2018 Poster: Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks »
Yusuke Tsuzuku · Issei Sato · Masashi Sugiyama -
2018 Poster: Monte-Carlo Tree Search for Constrained POMDPs »
Jongmin Lee · Geon-Hyeong Kim · Pascal Poupart · Kee-Eung Kim -
2018 Poster: Masking: A New Perspective of Noisy Supervision »
Bo Han · Jiangchao Yao · Gang Niu · Mingyuan Zhou · Ivor Tsang · Ya Zhang · Masashi Sugiyama -
2018 Poster: Co-teaching: Robust training of deep neural networks with extremely noisy labels »
Bo Han · Quanming Yao · Xingrui Yu · Gang Niu · Miao Xu · Weihua Hu · Ivor Tsang · Masashi Sugiyama -
2017 Poster: Positive-Unlabeled Learning with Non-Negative Risk Estimator »
Ryuichi Kiryo · Gang Niu · Marthinus C du Plessis · Masashi Sugiyama -
2017 Poster: Learning from Complementary Labels »
Takashi Ishida · Gang Niu · Weihua Hu · Masashi Sugiyama -
2017 Oral: Positive-Unlabeled Learning with Non-Negative Risk Estimator »
Ryuichi Kiryo · Gang Niu · Marthinus C du Plessis · Masashi Sugiyama -
2017 Poster: Expectation Propagation for t-Exponential Family Using q-Algebra »
Futoshi Futami · Issei Sato · Masashi Sugiyama -
2016 Poster: Efficient Neural Codes under Metabolic Constraints »
Zhuo Wang · Xue-Xin Wei · Alan A Stocker · Daniel Lee -
2016 Poster: Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning »
Gang Niu · Marthinus Christoffel du Plessis · Tomoya Sakai · Yao Ma · Masashi Sugiyama -
2016 Poster: Maximizing Influence in an Ising Network: A Mean-Field Optimal Solution »
Christopher W Lynn · Daniel Lee -
2014 Workshop: Novel Trends and Applications in Reinforcement Learning »
Csaba Szepesvari · Marc Deisenroth · Sergey Levine · Pedro Ortega · Brian Ziebart · Emma Brunskill · Naftali Tishby · Gerhard Neumann · Daniel Lee · Sridhar Mahadevan · Pieter Abbeel · David Silver · Vicenç Gómez -
2014 Poster: Analysis of Variational Bayesian Latent Dirichlet Allocation: Weaker Sparsity Than MAP »
Shinichi Nakajima · Issei Sato · Masashi Sugiyama · Kazuho Watanabe · Hiroko Kobayashi -
2014 Poster: Multitask learning meets tensor factorization: task imputation via convex optimization »
Kishan Wimalawarne · Masashi Sugiyama · Ryota Tomioka -
2014 Poster: Analysis of Learning from Positive and Unlabeled Data »
Marthinus C du Plessis · Gang Niu · Masashi Sugiyama -
2013 Poster: Parametric Task Learning »
Ichiro Takeuchi · Tatsuya Hongo · Masashi Sugiyama · Shinichi Nakajima -
2013 Poster: Global Solver and Its Efficient Approximation for Variational Bayesian Low-rank Subspace Clustering »
Shinichi Nakajima · Akiko Takeda · S. Derin Babacan · Masashi Sugiyama · Ichiro Takeuchi -
2013 Poster: Optimal Neural Population Codes for High-dimensional Stimulus Variables »
Zhuo Wang · Alan A Stocker · Daniel Lee -
2012 Poster: Cost-Sensitive Exploration in Bayesian Reinforcement Learning »
Dongho Kim · Kee-Eung Kim · Pascal Poupart -
2012 Poster: Nonparametric Bayesian Inverse Reinforcement Learning for Multiple Reward Functions »
Jaedeug Choi · Kee-Eung Kim -
2012 Poster: Optimal Neural Tuning Curves for Arbitrary Stimulus Distributions: Discrimax, Infomax and Minimum $L_p$ Loss »
Zhuo Wang · Alan A Stocker · Daniel Lee -
2012 Poster: Diffusion Decision Making for Adaptive k-Nearest Neighbor Classification »
Yung-Kyun Noh · Frank Park · Daniel Lee -
2012 Poster: Perfect Dimensionality Recovery by Variational Bayesian PCA »
Shinichi Nakajima · Ryota Tomioka · Masashi Sugiyama · S. Derin Babacan -
2012 Poster: Density-Difference Estimation »
Masashi Sugiyama · Takafumi Kanamori · Taiji Suzuki · Marthinus C du Plessis · Song Liu · Ichiro Takeuchi -
2011 Poster: Relative Density-Ratio Estimation for Robust Distribution Comparison »
Makoto Yamada · Taiji Suzuki · Takafumi Kanamori · Hirotaka Hachiya · Masashi Sugiyama -
2011 Poster: Target Neighbor Consistent Feature Weighting for Nearest Neighbor Classification »
Ichiro Takeuchi · Masashi Sugiyama -
2011 Poster: Analysis and Improvement of Policy Gradient Estimation »
Tingting Zhao · Hirotaka Hachiya · Gang Niu · Masashi Sugiyama -
2011 Poster: Global Solution of Fully-Observed Variational Bayesian Matrix Factorization is Column-Wise Independent »
Shinichi Nakajima · Masashi Sugiyama · S. Derin Babacan -
2011 Poster: MAP Inference for Bayesian Inverse Reinforcement Learning »
Jaedeug Choi · Kee-Eung Kim -
2010 Spotlight: Global Analytic Solution for Variational Bayesian Matrix Factorization »
Shinichi Nakajima · Masashi Sugiyama · Ryota Tomioka -
2010 Poster: Learning via Gaussian Herding »
Yacov Crammer · Daniel Lee -
2010 Poster: Global Analytic Solution for Variational Bayesian Matrix Factorization »
Shinichi Nakajima · Masashi Sugiyama · Ryota Tomioka -
2010 Poster: Generative Local Metric Learning for Nearest Neighbor Classification »
Yung-Kyun Noh · Byoung-Tak Zhang · Daniel Lee -
2008 Poster: Extended Grassmann Kernels for Subspace-Based Learning »
Jihun Hamm · Daniel Lee -
2008 Poster: Efficient Direct Density Ratio Estimation for Non-stationarity Adaptation and Outlier Detection »
Takafumi Kanamori · Shohei Hido · Masashi Sugiyama -
2007 Oral: Blind channel identification for speech dereverberation using l1-norm sparse learning »
Yuanqing Lin · Jingdong Chen · Youngmoo E Kim · Daniel Lee -
2007 Poster: Blind channel identification for speech dereverberation using l1-norm sparse learning »
Yuanqing Lin · Jingdong Chen · Youngmoo E Kim · Daniel Lee -
2007 Poster: Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation »
Masashi Sugiyama · Shinichi Nakajima · Hisashi Kashima · Paul von Buenau · Motoaki Kawanabe -
2007 Poster: Multi-Task Learning via Conic Programming »
Tsuyoshi Kato · Hisashi Kashima · Masashi Sugiyama · Kiyoshi Asai -
2006 Workshop: Learning when test and training inputs have different distributions »
Joaquin Quiñonero Candela · Masashi Sugiyama · Anton Schwaighofer · Neil D Lawrence -
2006 Poster: Mixture Regression for Covariate Shift »
Amos Storkey · Masashi Sugiyama