Timezone: »

 
Unsupervised Difficulty Estimation
Octavio Arriaga · Matias Valdenegro-Toro

Mon Dec 07 10:11 AM -- 10:21 AM (PST) @

"Evaluating difficulty and biases in machine learning models has become of ex- treme importance as current models are now being applied in real-world situations. In this paper we present a simple method for calculating a difficulty score based on the accumulation of losses for each sample during training. Our proposed method does not require any modification of the model neither any external supervision. We test and analyze our approach in two different settings that provide empirical evidence of the applicability of our method."

Author Information

Octavio Arriaga (University of Bremen)
Matias Valdenegro-Toro (German Research Center for Artificial Intelligence)

More from the Same Authors

  • 2020 : Automatic Detection and Classification of Tick-borne Skin Lesions using Deep Learning »
    Matias Valdenegro-Toro
  • 2021 : Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings »
    Matias Valdenegro-Toro
  • 2021 : Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning »
    Aaqib Parvez Mohammed · Matias Valdenegro-Toro
  • 2021 : Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning »
    Aaqib Parvez Mohammed · Matias Valdenegro-Toro
  • 2021 : Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings »
    Matias Valdenegro-Toro
  • 2021 : Q&A Oral presentations »
    Matias Valdenegro-Toro · Andres Munoz · Johan Obando Ceron · Anil Batra
  • 2021 : Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings »
    Matias Valdenegro-Toro
  • 2020 : QA Long Presentation II »
    Matias Valdenegro-Toro · Gefersom Lima · Nicolas Araque · Matías Molina
  • 2019 : Poster session »
    Sebastian Farquhar · Erik Daxberger · Andreas Look · Matt Benatan · Ruiyi Zhang · Marton Havasi · Fredrik Gustafsson · James A Brofos · Nabeel Seedat · Micha Livne · Ivan Ustyuzhaninov · Adam Cobb · Felix D McGregor · Patrick McClure · Tim R. Davidson · Gaurush Hiranandani · Sanjeev Arora · Masha Itkina · Didrik Nielsen · William Harvey · Matias Valdenegro-Toro · Stefano Peluchetti · Riccardo Moriconi · Tianyu Cui · Vaclav Smidl · Taylan Cemgil · Jack Fitzsimons · He Zhao · · mariana vargas vieyra · Apratim Bhattacharyya · Rahul Sharma · Geoffroy Dubourg-Felonneau · Jonathan Warrell · Slava Voloshynovskiy · Mihaela Rosca · Jiaming Song · Andrew Ross · Homa Fashandi · Ruiqi Gao · Hooshmand Shokri Razaghi · Joshua Chang · Zhenzhong Xiao · Vanessa Boehm · Giorgio Giannone · Ranganath Krishnan · Joe Davison · Arsenii Ashukha · Jeremiah Liu · Sicong (Sheldon) Huang · Evgenii Nikishin · Sunho Park · Nilesh Ahuja · Mahesh Subedar · · Artyom Gadetsky · Jhosimar Arias Figueroa · Tim G. J. Rudner · Waseem Aslam · Adrián Csiszárik · John Moberg · Ali Hebbal · Kathrin Grosse · Pekka Marttinen · Bang An · Hlynur Jónsson · Samuel Kessler · Abhishek Kumar · Mikhail Figurnov · Omesh Tickoo · Steindor Saemundsson · Ari Heljakka · Dániel Varga · Niklas Heim · Simone Rossi · Max Laves · Waseem Gharbieh · Nicholas Roberts · Luis Armando Pérez Rey · Matthew Willetts · Prithvijit Chakrabarty · Sumedh Ghaisas · Carl Shneider · Wray Buntine · Kamil Adamczewski · Xavier Gitiaux · Suwen Lin · Hao Fu · Gunnar Rätsch · Aidan Gomez · Erik Bodin · Dinh Phung · Lennart Svensson · Juliano Tusi Amaral Laganá Pinto · Milad Alizadeh · Jianzhun Du · Kevin Murphy · Beatrix Benkő · Shashaank Vattikuti · Jonathan Gordon · Christopher Kanan · Sontje Ihler · Darin Graham · Michael Teng · Louis Kirsch · Tomas Pevny · Taras Holotyak