Timezone: »

 
Poster
Attribution-Based Confidence Metric For Deep Neural Networks
Susmit Jha · Sunny Raj · Steven Fernandes · Sumit K Jha · Somesh Jha · Brian Jalaian · Gunjan Verma · Ananthram Swami

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #105
in Applications -- Fairness, Accountability, and Transparency »

We propose a novel confidence metric, namely, attribution-based confidence (ABC) for deep neural networks (DNNs). ABC metric characterizes whether the output of a DNN on an input can be trusted. DNNs are known to be brittle on inputs outside the training distribution and are, hence, susceptible to adversarial attacks. This fragility is compounded by a lack of effectively computable measures of model confidence that correlate well with the accuracy of DNNs. These factors have impeded the adoption of DNNs in high-assurance systems. The proposed ABC metric addresses these challenges. It does not require access to the training data, the use of ensembles, or the need to train a calibration model on a held-out validation set. Hence, the new metric is usable even when only a trained model is available for inference. We mathematically motivate the proposed metric and evaluate its effectiveness with two sets of experiments. First, we study the change in accuracy and the associated confidence over out-of-distribution inputs. Second, we consider several digital and physically realizable attacks such as FGSM, CW, DeepFool, PGD, and adversarial patch generation methods. The ABC metric is low on out-of-distribution data and adversarial examples, where the accuracy of the model is also low. These experiments demonstrate the effectiveness of the ABC metric to make DNNs more trustworthy and resilient.

Keywords: Fairness, Accountability, and Transparency · Visualization or Exposition Techniques for Deep Networks · Applications · Algorithms -> Uncertainty Estimation; Deep Learning
[ Paper]  [ Slides] 

Author Information

Susmit Jha (SRI)

Susmit Jha is a Principal Computer Scientist in the Computer Science Laboratory at SRI International where he is the principal investigator for DARPA Assured Autonomy, DARPA Symbiotic Design of CPS, DARPA Intent-driven Design of Adaptive Systems, IARPA TrojAI, US ARL’s Principles of Robust Learning in IoBT CRA, and NSF Self-improving Cyberphysical Systems. Prior to joining SRI, Dr. Jha was a Staff Scientist at UTRC (Raytheon Technologies), Berkeley, and a Research Scientist at Intel. Dr. Jha received his Ph.D. in Electrical Engineering and Computer Science from the University of California, Berkeley in 2011.

Sunny Raj (University of Central Florida)
Steven Fernandes (University of Central Florida)
Sumit K Jha (University of Central Florida)

Dr. Sumit K. Jha is an Associate Professor of Computer Science at the University of Central Florida (UCF), Orlando. Dr. Jha joined the University of Central Florida in 2010 after receiving his Ph.D. in Computer Science at Carnegie Mellon University. Before joining Carnegie Mellon, he graduated with B.Tech (Honors) in Computer Science and Engineering from the Indian Institute of Technology Kharagpur in 2004. Dr. Jha has worked on R&D problems at Microsoft Research India, General Motors, INRIA France and the Air Force Research Lab Information Directorate. His research has been supported by the National Science Foundation, the Air Force Office of Scientific Research, the Oak Ridge National Laboratory, the Royal Bank of Canada, the Florida Center for Cybersecurity, and the Air Force Research Laboratory. He is a full member of the Sigma Xi and is a recipient of the IEEE Orlando Engineering Educator Excellence Award. Dr. Jha was awarded the prestigious Air Force Young Investigator Award in 2016 and his research has led to three Best Paper awards.

Somesh Jha (University of Wisconsin, Madison)
Brian Jalaian (U.S. Army Research Laboratory)
Gunjan Verma (U.S. Army Research Laboratory)
Ananthram Swami (Army Research Laboratory, Adelphi)

More from the Same Authors