ItemConference Paper

Co Adaptation Vs Signal Altering Regularization Layer in Deep Learning: A Trade Off Analysis via Node Redundancy and Transfer Learning

Title
Co Adaptation Vs Signal Altering Regularization Layer in Deep Learning: A Trade Off Analysis via Node Redundancy and Transfer Learning
Creator
Omathil, Gelesh G.; Sreeja, C.S.
Description
This study investigates the trade-off between incorporating regularization layers-such as Dropout, R-Drop, and Gaussian Dropout-and the phenomenon of co-adaptation in deep learning models. While regularization is designed to enhance generalization by disrupting hidden layer activations and reducing overfitting, it may also introduce node redundancy, potentially diminishing the model's capacity to learn efficiently. Conversely, co-adaptation, though often considered undesirable, may help preserve beneficial internal representations that contribute to learning generalizable data patterns-particularly in transfer learning scenarios-where regularization may inadvertently hinder such learning. Using the CIFAR-10 dataset, this study conducts an empirical analysis of how various regularization strategies influence neuron redundancy and downstream transfer performance. The results indicate that, although regularization effectively controls overfitting, excessive distortion in hidden representations can impair the model's ability to generalize across tasks. These findings provide insights into the need for balanced regularization strategies that maintain useful structure while minimizing detrimental redundancy. 2025 IEEE.
Source
INDISCON 2025 - IEEE 6th India Council International Subsections Conference, Proceedings;
Date
01-01-2025
Publisher
Institute of Electrical and Electronics Engineers Inc.
Subject
Deep learning; Gaussian Dropout; Node redundancy; Overfitting; RDrop; Regularization; Regularization Layer; Transfer learning
Coverage
Omathil G.G., Christ University, Department of Computer Science, Bangalore, India; Sreeja C.S., Christ University, Department of Computer Science, Bangalore, India
Rights
Restricted Access; Hardcopy may be available in the library
Relation
ISBN: 979-833151504-1;
Format
online
Language
English
Type
Conference paper

Citation

Omathil, Gelesh G.; Sreeja, C.S., “Co Adaptation Vs Signal Altering Regularization Layer in Deep Learning: A Trade Off Analysis via Node Redundancy and Transfer Learning,” CHRIST (Deemed To Be University) Institutional Repository, accessed June 18, 2026, https://archives.christuniversity.in/items/show/26170.

Output Formats