Boundary between noise and information applied to filtering neural network weight matrices
Deep neural networks have been successfully applied to a broad range of problems where overparametrization yields weight matrices which are partially random. A comparison of weight matrix singular vectors to the Porter-Thomas distribution suggests that there is a boundary between randomness and learned information in the singular value spectrum. Inspired by this finding, we introduce an algorithm for noise filtering, which both removes small singular values and reduces the magnitude of large singular values to counteract the effect of level repulsion between the noise and the information part of the spectrum. For networks trained in the presence of label noise, we indeed find that the generalization performance improves significantly due to noise filtering.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Polishing Decision-Based Adversarial Noise With a Customized Sampling
As an effective black-box adversarial attack, decision-based methods polish adversarial noise by querying the target model. Among them, boundary attack is widely applied due to its powerful noise compression capability, …
Adversarial Attackimage-classificationImage ClassificationCombined Approach for Image Segmentation
Many image segmentation techniques have been developed over the past two decades for segmenting the images, which help for object recognition, occlusion boundary estimation within motion or stereo systems, image compress…
ClusteringDenoisingImage CompressionImage Segmentation+3Boundary-aware Contrastive Learning for Semi-supervised Nuclei Instance Segmentation
Semi-supervised segmentation methods have demonstrated promising results in natural scenarios, providing a solution to reduce dependency on manual annotation. However, these methods face significant challenges when direc…
Contrastive LearningDenoisingInstance SegmentationSegmentation+2Noise-Aware Boundary-Enhanced Generative Learning for Ultrasound Speckle Reduction
Ultrasound is a non-invasive, real-time, and cost-effective imaging technique widely used in clinical diagnosis. However, its diagnostic efficacy is often compromised by inherent speckle noise that degrades image quality…
LiDAR Ground Filtering Algorithm for Urban Areas Using Scan Line Based Segmentation
This paper addresses the task of separating ground points from airborne LiDAR point cloud data in urban areas. A novel ground filtering method using scan line segmentation is proposed here, which we call SLSGF. It utiliz…
Denoising