Deep learning visualizer
Dropout Layer Visualizer
See how randomly disabling neurons reduces overfitting.
Layer8 neuronsDropout rate0.40Active6ModeTraining
Input activations (4 values)
Training Pass
Pass 4 / 12Random Mask Applied
| Original activations | 1.1 | 0.73 | 0.97 | 0.56 | 0.64 | 0.97 | 0.57 | 1.03 |
|---|---|---|---|---|---|---|---|---|
| Dropout mask | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 |
| Scaled output | 1.84 | 1.22 | 1.62 | 0.94 | 1.06 | 1.62 | 0 | 0 |
y = (x x mask) / (1 - p)neuron 1: 1.1 x 1 / 0.6 = 1.84Training
Different mask each passSame expected activation
Inference
Why dropout works
- Randomly dropping neurons prevents co-adaptations.
- The network learns redundant, more robust features.
- At inference, all neurons contribute to the prediction.
- This reduces overfitting and improves generalization.
Regularization effect: dropout approximates training an ensemble of many thinned networks and averaging their predictions.
import torch.nn as nndropout = nn.Dropout(p=0.4)model.train() # dropout is activey_train = dropout(hidden)model.eval() # dropout is disabled