You are training a deep neural network, and you want to implement a regularization technique to prevent overfitting. During each training epoch, this method randomly deactivates a pre-determined percentage of neurons, forcing the network to learn redundant representations. What is this technique called?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's a cool trick to stop your deep networks from getting lazy and overfitting. During training, we can randomly shut off a set percentage of the neurons in a layer on every single pass. This forces the remaining neurons to step up and learn the features, rather than relying on one or two super-neurons to do all the work. We call this technique Dropout. (Now, sometimes you'll see this translated weirdly in non-English materials as "abandon" or "give up", but don't let that throw you off — the technical industry term you need to know is Dropout.) It makes your network much more robust and prevents overfitting.
Full explanation below image
Full Explanation
Dropout is a widely used regularization technique in deep learning designed to mitigate overfitting. During training, dropout randomly deactivates (sets to zero) a specified fraction of neurons in a layer at each step of gradient descent. This fraction is determined by a hyperparameter, typically set between 0.2 and 0.5.
By randomly disabling neurons, dropout prevents co-adaptation of features. In a standard network, neurons can develop codependency during training, where some neurons compensate for the errors of others. Dropout breaks these dependencies because a neuron cannot rely on the presence of any specific neighboring neuron. Consequently, the network is forced to learn more robust, redundant features that generalize better to unseen data. At test time, all neurons are active, but their outputs are scaled down by the dropout rate to match the expected values during training.
Let's look at the distractors: - Early stopping (Early termination) stops the training process entirely once the validation loss starts to rise, rather than modifying the active neurons within epochs. - Cross-validation (Validation croisée) is a model evaluation technique where the dataset is partitioned into multiple folds to test generalizability; it does not change the network's internal structure during training. - Batch normalization scales the activations of a layer to have a mean of zero and variance of one to accelerate training stability; it is not primarily a regularization technique designed to deactivate neurons.
Therefore, Dropout (Option B) is the correct answer.