Papers L2 Regularization
“L2 Regularization” 태그가 달린 논문 128편 · 필터 해제
Overcoming catastrophic forgetting in neural networks
Catastrophic forgetting is the primary challenge that hinders continual learning, which refers to a neural network ability to sequentially learn multiple tasks while retaining previously acquired knowledge. Elastic Weigh…
Continual LearningL2 RegularizationLifelong learningFrom large-eddy simulations to deep learning: A U-net model for fast urban canopy flow predictions
Accurate prediction of wind flow fields in urban canopies is crucial for ensuring pedestrian comfort, safety, and sustainable urban design. Traditional methods using wind tunnels and Computational Fluid Dynamics, such as…
GPUL2 RegularizationDACN: Dual-Attention Convolutional Network for Hyperspectral Image Super-Resolution
2D convolutional neural networks (CNNs) have attracted significant attention for hyperspectral image super-resolution tasks. However, a key limitation is their reliance on local neighborhoods, which leads to a lack of gl…
Hyperspectral Image Super-ResolutionImage Super-ResolutionL2 RegularizationSuper-ResolutionGeometry of Learning -- L2 Phase Transitions in Deep and Shallow Neural Networks
When neural networks (NNs) are subject to L2 regularization, increasing the regularization strength beyond a certain threshold pushes the model into an under-parameterization regime. This transition manifests as a first-…
L2 RegularizationUnderstand the Effect of Importance Weighting in Deep Learning on Dataset Shift
We evaluate the effectiveness of importance weighting in deep neural networks under label shift and covariate shift. On synthetic 2D data (linearly separable and moon-shaped) using logistic regression and MLPs, we observ…
L2 RegularizationDeep Learning in Renewable Energy Forecasting: A Cross-Dataset Evaluation of Temporal and Spatial Models
Unpredictability of renewable energy sources coupled with the complexity of those methods used for various purposes in this area calls for the development of robust methods such as DL models within the renewable energy d…
Hyperparameter OptimizationL2 RegularizationSemantic segmentation for building houses from wooden cubes
Automated construction is one of the most promising areas that can improve efficiency, reduce costs and minimize errors in the process of building construction. In this paper, a comparative analysis of three neural netwo…
L2 RegularizationSemantic SegmentationGPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA
We explore the potential of integrating learnable and interpretable modules--specifically Kolmogorov-Arnold Networks (KAN) and graph-based representations--within a pre-trained GPT-2 model to enhance multi-task learning …
Graph AttentionKolmogorov-Arnold NetworksL2 RegularizationMulti-Task Learning+2CtrTab: Tabular Data Synthesis with High-Dimensional and Limited Data
Diffusion-based tabular data synthesis models have yielded promising results. However, we observe that when the data dimensionality increases, existing models tend to degenerate and may perform even worse than simpler, n…
DiversityL2 RegularizationLow-rank bias, weight decay, and model merging in neural networks
We explore the low-rank structure of the weight matrices in neural networks originating from training with Gradient Descent (GD) and Gradient Flow (GF) with $L2$ regularization (also known as weight decay). We show sever…
L2 RegularizationMultimodal Bearing Fault Classification Under Variable Conditions: A 1D CNN with Transfer Learning
Bearings play an integral role in ensuring the reliability and efficiency of rotating machinery - reducing friction and handling critical loads. Bearing failures that constitute up to 90% of mechanical faults highlight t…
Fault DetectionFrictionL2 RegularizationTransfer LearningRenewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis
The increasing focus on predicting renewable energy production aligns with advancements in deep learning (DL). The inherent variability of renewable sources and the complexity of prediction methods require robust approac…
Hyperparameter OptimizationL2 RegularizationLearning in Log-Domain: Subthreshold Analog AI Accelerator Based on Stochastic Gradient Descent
The rapid proliferation of AI models, coupled with growing demand for edge deployment, necessitates the development of AI hardware that is both high-performance and energy-efficient. In this paper, we propose a novel ana…
L2 RegularizationSuper-Resolution for Remote Sensing Imagery via the Coupling of a Variational Model and Deep Learning
Image super-resolution (SR) is an effective way to enhance the spatial resolution and detail information of remote sensing images, to obtain a superior visual quality. As SR is severely ill-conditioned, effective image p…
Image Super-ResolutionL2 RegularizationMulti-Frame Super-ResolutionSuper-ResolutionParkinson's Disease Diagnosis Through Deep Learning: A Novel LSTM-Based Approach for Freezing of Gait Detection
Deep learning holds tremendous potential in healthcare for uncovering hidden patterns within extensive clinical datasets, aiding in the diagnosis of various diseases. Parkinson's disease (PD) is a neurodegenerative condi…
Feature EngineeringL2 RegularizationSpecificityEffectiveness of L2 Regularization in Privacy-Preserving Machine Learning
Artificial intelligence, machine learning, and deep learning as a service have become the status quo for many industries, leading to the widespread deployment of models that handle sensitive data. Well-performing models,…
Inference AttackL2 RegularizationMembership Inference AttackPrivacy PreservingAnalysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm
Semi-supervised learning (SSL) is a machine learning methodology that leverages unlabeled data in conjunction with a limited amount of labeled data. Although SSL has been applied in various applications and its effective…
Binary ClassificationL2 RegularizationRecurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling
Recurrent stochastic configuration networks (RSCNs) have shown great potential in modelling nonlinear dynamic systems with uncertainties. This paper presents an RSCN with hybrid regularization to enhance both the learnin…
L2 RegularizationCarbon price fluctuation prediction using blockchain information A new hybrid machine learning approach
In this study, the novel hybrid machine learning approach is proposed in carbon price fluctuation prediction. Specifically, a research framework integrating DILATED Convolutional Neural Networks (CNN) and Long Short-Term…
Hybrid Machine LearningL2 RegularizationWeight decay induces low-rank attention layers
The effect of regularizers such as weight decay when training deep neural networks is not well understood. We study the influence of weight decay as well as $L2$-regularization when training neural network models in whic…
L2 RegularizationLanguage Modelling