Timezone: »

 
Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning
Aaqib Parvez Mohammed · Matias Valdenegro-Toro
Event URL: https://openreview.net/forum?id=5qaXOgqsNAV »

Reinforcement Learning (RL) based solutions are being adopted in a variety of domains including robotics, health care and industrial automation. Most focus is given to when these solutions work well, but they fail when presented with out of distribution inputs. RL policies share the same faults as most machine learning models. Out of distribution detection for RL is generally not well covered in the literature, and there is a lack of benchmarks for this task. In this work we propose a benchmark to evaluate OOD detection methods in a Reinforcement Learning setting, by modifying the physical parameters of non-visual standard environments or corrupting the state observation for visual environments. We discuss ways to generate custom RL environments that can produce OOD data, and evaluate three uncertainty methods for the OOD detection task. Our results show that ensemble methods have the best OOD detection performance with a lower standard deviation across multiple environments.

Author Information

Aaqib Parvez Mohammed (Hochschule Bonn-Rhein-Sieg)
Matias Valdenegro-Toro (German Research Center for Artificial Intelligence)

More from the Same Authors

  • 2020 : Automatic Detection and Classification of Tick-borne Skin Lesions using Deep Learning »
    Matias Valdenegro-Toro
  • 2021 : Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings »
    Matias Valdenegro-Toro
  • 2021 : Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning »
    Aaqib Parvez Mohammed · Matias Valdenegro-Toro
  • 2021 : Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings »
    Matias Valdenegro-Toro
  • 2021 : Q&A Oral presentations »
    Matias Valdenegro-Toro · Andres Munoz · Johan Obando Ceron · Anil Batra
  • 2021 : Exploring the Limits of Epistemic Uncertainty Quantification in Low-Shot Settings »
    Matias Valdenegro-Toro
  • 2020 : QA Long Presentation II »
    Matias Valdenegro-Toro · Gefersom Lima · Nicolas Araque · Matías Molina
  • 2020 : Unsupervised Difficulty Estimation »
    Octavio Arriaga · Matias Valdenegro-Toro
  • 2019 : Poster session »
    Sebastian Farquhar · Erik Daxberger · Andreas Look · Matt Benatan · Ruiyi Zhang · Marton Havasi · Fredrik Gustafsson · James A Brofos · Nabeel Seedat · Micha Livne · Ivan Ustyuzhaninov · Adam Cobb · Felix D McGregor · Patrick McClure · Tim R. Davidson · Gaurush Hiranandani · Sanjeev Arora · Masha Itkina · Didrik Nielsen · William Harvey · Matias Valdenegro-Toro · Stefano Peluchetti · Riccardo Moriconi · Tianyu Cui · Vaclav Smidl · Taylan Cemgil · Jack Fitzsimons · He Zhao · · mariana vargas vieyra · Apratim Bhattacharyya · Rahul Sharma · Geoffroy Dubourg-Felonneau · Jonathan Warrell · Slava Voloshynovskiy · Mihaela Rosca · Jiaming Song · Andrew Ross · Homa Fashandi · Ruiqi Gao · Hooshmand Shokri Razaghi · Joshua Chang · Zhenzhong Xiao · Vanessa Boehm · Giorgio Giannone · Ranganath Krishnan · Joe Davison · Arsenii Ashukha · Jeremiah Liu · Sicong (Sheldon) Huang · Evgenii Nikishin · Sunho Park · Nilesh Ahuja · Mahesh Subedar · · Artyom Gadetsky · Jhosimar Arias Figueroa · Tim G. J. Rudner · Waseem Aslam · Adrián Csiszárik · John Moberg · Ali Hebbal · Kathrin Grosse · Pekka Marttinen · Bang An · Hlynur Jónsson · Samuel Kessler · Abhishek Kumar · Mikhail Figurnov · Omesh Tickoo · Steindor Saemundsson · Ari Heljakka · Dániel Varga · Niklas Heim · Simone Rossi · Max Laves · Waseem Gharbieh · Nicholas Roberts · Luis Armando Pérez Rey · Matthew Willetts · Prithvijit Chakrabarty · Sumedh Ghaisas · Carl Shneider · Wray Buntine · Kamil Adamczewski · Xavier Gitiaux · Suwen Lin · Hao Fu · Gunnar Rätsch · Aidan Gomez · Erik Bodin · Dinh Phung · Lennart Svensson · Juliano Tusi Amaral Laganá Pinto · Milad Alizadeh · Jianzhun Du · Kevin Murphy · Beatrix Benkő · Shashaank Vattikuti · Jonathan Gordon · Christopher Kanan · Sontje Ihler · Darin Graham · Michael Teng · Louis Kirsch · Tomas Pevny · Taras Holotyak