Skip to yearly menu bar Skip to main content


Poster

Towards the Difficulty for a Deep Neural Network to Learn Concepts of Different Complexities

Dongrui Liu · Huiqi Deng · Xu Cheng · Xu Cheng · Qihan Ren · Kangrui Wang · Quanshi Zhang

Great Hall & Hall B1+B2 (level 1) #1523

Abstract:

This paper theoretically explains the intuition that simple concepts are more likely to be learned by deep neural networks (DNNs) than complex concepts. In fact, recent studies have observed [24, 15] and proved [26] the emergence of interactive concepts in a DNN, i.e., it is proven that a DNN usually only encodes a small number of interactive concepts, and can be considered to use their interaction effects to compute inference scores. Each interactive concept is encoded by the DNN to represent the collaboration between a set of input variables. Therefore, in this study, we aim to theoretically explain that interactive concepts involving more input variables (i.e., more complex concepts) are more difficult to learn. Our finding clarifies the exact conceptual complexity that boosts the learning difficulty.

Chat is not available.