Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN
In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover that the confusing samples of a DNN, which are represented by non-generalizable interactions, are determined by its low-layer parameters. In comparison, other factors, such as high-layer parameters and network architecture, have much less impact on the composition of confusing samples. Two DNNs with different low-layer parameters usually have fully different sets of confusing samples, even though they have similar performance. This finding extends the understanding of the lottery ticket hypothesis, and well explains distinctive representation power of different DNNs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
M$^5$L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking
Classifying the confusing samples in the course of RGBT tracking is a quite challenging problem, which hasn't got satisfied solution. Existing methods only focus on enlarging the boundary between positive and negative sa…
Metric LearningDomain Compression and its Application to Randomness-Optimal Distributed Goodness-of-Fit
We study goodness-of-fit of discrete distributions in the distributed setting, where samples are divided between multiple users who can only release a limited amount of information about their samples due to various info…
Task Understanding from Confusing Multi-task Data
Beyond machine learning's success in the specific tasks, research for learning multiple tasks simultaneously is referred to as multi-task learning. However, existing multi-task learning needs manual definition of tasks a…
Multi-Task LearningSemi-supervised Object Detection via Virtual Category Learning
Due to the costliness of labelled data in real-world applications, semi-supervised object detectors, underpinned by pseudo labelling, are appealing. However, handling confusing samples is nontrivial: discarding valuable …
Objectobject-detectionObject DetectionSemi-Supervised Object DetectionSemi-supervised Object Detection via Virtual Category Learning
Due to the lack of large amounts of labelled data to learn rich-expressive features of objects, semi-supervised detectors powered by pseudo labelling techniques usually make a tentative decision for the pseudo labels of …
Objectobject-detectionObject DetectionSemi-Supervised Object Detection